Max level shown:
12025-10-03T21:15:01.222ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 573 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 6773c0c6-0b81-4042-a18b-4f8a806e1e76 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
22025-10-03T21:15:05.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 96 reason = Timeout
32025-10-03T21:15:05.189ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42025-10-03T21:15:05.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52025-10-03T21:15:05.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62025-10-03T21:15:05.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
72025-10-03T21:15:05.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82025-10-03T21:15:05.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
92025-10-03T21:15:05.195ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
102025-10-03T21:15:05.195ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.503813ms iteration = 96 status = {"config_updated":false}
112025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
122025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
132025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
142025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
152025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
162025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
172025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
182025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
192025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
202025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
212025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
222025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
232025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
242025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
252025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
262025-10-03T21:15:10.188ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-10-03 21:05:10.188539157 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
272025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
282025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 17 reason = Timeout
292025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
302025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
312025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
322025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
332025-10-03T21:15:10.188ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
342025-10-03T21:15:10.188ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = ExternalNetworking step id = Ensure
352025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
362025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 9 reason = Timeout
372025-10-03T21:15:10.188ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
382025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
392025-10-03T21:15:10.189ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("486849a9-1733-4aa9-9b66-43ace2da0eea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 4955, tv_nsec: 679811214 } })
402025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
412025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
422025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
432025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 20 reason = Timeout
442025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
452025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
462025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
472025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
482025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
492025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 17 reason = Timeout
502025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.74\u{b5}s iteration = 17 status = {"disabled":true,"errors":[],"sps":[]}
512025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
522025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
532025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
542025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
552025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
562025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
572025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
582025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
592025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
602025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
612025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
622025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
632025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
642025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
652025-10-03T21:15:10.189ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
662025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
672025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
682025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
692025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
702025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
712025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
722025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
732025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
742025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
752025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 9 reason = Timeout
762025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
772025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
782025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
792025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
802025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 97 reason = Timeout
812025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
822025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
832025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
842025-10-03T21:15:10.189ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2efcfb05-0c3d-45aa-b120-c334f3706be4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a7c1656-d513-4cc3-b2b0-8e31ad2b5545.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 4955, tv_nsec: 680359570 } })
852025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
862025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
872025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
882025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
892025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
902025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
912025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
922025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
932025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
942025-10-03T21:15:10.189ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
952025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
962025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
972025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
982025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
992025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1002025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
1012025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
1022025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1032025-10-03T21:15:10.190ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1042025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1052025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
1062025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1072025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1082025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
1092025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1102025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1112025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.802441ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
1122025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1132025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1142025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1152025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
1162025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1172025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1182025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1192025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
1202025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
1212025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
1222025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1232025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
1242025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.039383ms iteration = 9 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
1252025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
1262025-10-03T21:15:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1272025-10-03T21:15:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1282025-10-03T21:15:10.194ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1292025-10-03T21:15:10.194ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
1302025-10-03T21:15:10.194ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 5.698394ms iteration = 9 status = {"by_rx":{},"error":null}
1312025-10-03T21:15:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1322025-10-03T21:15:10.195ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 4955, tv_nsec: 685906813 } })
1332025-10-03T21:15:10.195ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
1342025-10-03T21:15:10.198ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-10-03 21:05:10.188539157 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1352025-10-03T21:15:10.198ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.863611ms iteration = 9 status = {"expiration":"2025-10-03T21:05:10.188539157Z","pruned":[]}
1362025-10-03T21:15:10.198ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1372025-10-03T21:15:10.198ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1382025-10-03T21:15:10.199ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
1392025-10-03T21:15:10.199ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
1402025-10-03T21:15:10.199ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 9.950772ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
1412025-10-03T21:15:10.199ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 4955, tv_nsec: 690070290 } })
1422025-10-03T21:15:10.199ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.901813ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1432025-10-03T21:15:10.199ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1442025-10-03T21:15:10.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
1452025-10-03T21:15:10.199ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
1462025-10-03T21:15:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
1472025-10-03T21:15:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
1482025-10-03T21:15:10.201ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.903495ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
1492025-10-03T21:15:10.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1502025-10-03T21:15:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1512025-10-03T21:15:10.202ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1522025-10-03T21:15:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1532025-10-03T21:15:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1542025-10-03T21:15:10.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1552025-10-03T21:15:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1562025-10-03T21:15:10.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1572025-10-03T21:15:10.204ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1582025-10-03T21:15:10.205ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1592025-10-03T21:15:10.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1602025-10-03T21:15:10.206ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1612025-10-03T21:15:10.206ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1622025-10-03T21:15:10.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1632025-10-03T21:15:10.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
1642025-10-03T21:15:10.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1652025-10-03T21:15:10.208ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1662025-10-03T21:15:10.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1672025-10-03T21:15:10.208ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1682025-10-03T21:15:10.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1692025-10-03T21:15:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1702025-10-03T21:15:10.209ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1712025-10-03T21:15:10.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1722025-10-03T21:15:10.210ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1732025-10-03T21:15:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
1742025-10-03T21:15:10.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
1752025-10-03T21:15:10.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
1762025-10-03T21:15:10.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
1772025-10-03T21:15:10.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1782025-10-03T21:15:10.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1792025-10-03T21:15:10.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1802025-10-03T21:15:10.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1812025-10-03T21:15:10.212ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1822025-10-03T21:15:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
1832025-10-03T21:15:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1842025-10-03T21:15:10.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1852025-10-03T21:15:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1862025-10-03T21:15:10.213ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
1872025-10-03T21:15:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
1882025-10-03T21:15:10.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1892025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1902025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1912025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1922025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1932025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1942025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1952025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1962025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1972025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1982025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1992025-10-03T21:15:10.221ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2002025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2012025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2022025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2032025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2042025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2052025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2062025-10-03T21:15:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2072025-10-03T21:15:10.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
2082025-10-03T21:15:10.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
2092025-10-03T21:15:10.225ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 36.36141ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2102025-10-03T21:15:10.225ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
2112025-10-03T21:15:10.225ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 36.26579ms iteration = 97 status = {"config_updated":false}
2122025-10-03T21:15:10.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2132025-10-03T21:15:10.225ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2142025-10-03T21:15:10.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2152025-10-03T21:15:10.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2162025-10-03T21:15:10.231ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 41.722791ms iteration = 9 status = {"dispatched":[],"errors":[],"glob_version":"195.0.0","globs_reprocessed":{},"no_receivers":[]}
2172025-10-03T21:15:10.238ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2182025-10-03T21:15:10.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2192025-10-03T21:15:10.240ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 4955, tv_nsec: 731439257 } })
2202025-10-03T21:15:10.241ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
2212025-10-03T21:15:10.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2222025-10-03T21:15:10.242ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2232025-10-03T21:15:10.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2242025-10-03T21:15:10.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2252025-10-03T21:15:10.245ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2262025-10-03T21:15:10.245ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2272025-10-03T21:15:10.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2282025-10-03T21:15:10.245ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 55.892911ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2292025-10-03T21:15:10.248ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2302025-10-03T21:15:10.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2312025-10-03T21:15:10.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2322025-10-03T21:15:10.251ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2332025-10-03T21:15:10.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2342025-10-03T21:15:10.251ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2352025-10-03T21:15:10.251ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2362025-10-03T21:15:10.251ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 62.558616ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2372025-10-03T21:15:10.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2382025-10-03T21:15:10.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2392025-10-03T21:15:10.255ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2402025-10-03T21:15:10.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2412025-10-03T21:15:10.255ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2422025-10-03T21:15:10.255ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2432025-10-03T21:15:10.256ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2442025-10-03T21:15:10.256ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
2452025-10-03T21:15:10.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2462025-10-03T21:15:10.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2472025-10-03T21:15:10.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2482025-10-03T21:15:10.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
2492025-10-03T21:15:10.258ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2502025-10-03T21:15:10.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2512025-10-03T21:15:10.259ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2522025-10-03T21:15:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2532025-10-03T21:15:10.264ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2542025-10-03T21:15:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2552025-10-03T21:15:10.265ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
2562025-10-03T21:15:10.267ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2572025-10-03T21:15:10.269ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
2582025-10-03T21:15:10.269ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2592025-10-03T21:15:10.269ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-10-03 21:07:12.445842 UTC dns_group = internal generation = 1
2602025-10-03T21:15:10.269ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 81.399439ms iteration = 9 status = {"generation":1}
2612025-10-03T21:15:10.269ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
2622025-10-03T21:15:10.271ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2632025-10-03T21:15:10.271ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2642025-10-03T21:15:10.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2652025-10-03T21:15:10.272ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
2662025-10-03T21:15:10.283ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
2672025-10-03T21:15:10.286ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-10-03T21:07:04.185535Z, time_modified: 2025-10-03T21:07:04.185535Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }) }
2682025-10-03T21:15:10.286ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 98.222869ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]}},"default_endpoint":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
2692025-10-03T21:15:10.296ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
2702025-10-03T21:15:10.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2712025-10-03T21:15:10.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2722025-10-03T21:15:10.301ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
2732025-10-03T21:15:10.304ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
2742025-10-03T21:15:10.304ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2752025-10-03T21:15:10.306ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 117.111402ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2762025-10-03T21:15:10.306ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2772025-10-03T21:15:10.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2782025-10-03T21:15:10.308ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2792025-10-03T21:15:10.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
2802025-10-03T21:15:10.308ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 119.666561ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
2812025-10-03T21:15:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2822025-10-03T21:15:10.313ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2832025-10-03T21:15:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2842025-10-03T21:15:10.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2852025-10-03T21:15:10.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2862025-10-03T21:15:10.317ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
2872025-10-03T21:15:10.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2882025-10-03T21:15:10.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2892025-10-03T21:15:10.320ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2902025-10-03T21:15:10.320ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 130.905058ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2912025-10-03T21:15:10.321ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 131.886109ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
2922025-10-03T21:15:10.321ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
2932025-10-03T21:15:10.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
2942025-10-03T21:15:10.324ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
2952025-10-03T21:15:10.324ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 135.275067ms iteration = 49 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
2962025-10-03T21:15:10.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
2972025-10-03T21:15:10.335ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2982025-10-03T21:15:10.335ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2992025-10-03T21:15:10.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3002025-10-03T21:15:10.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3012025-10-03T21:15:10.337ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
3022025-10-03T21:15:10.338ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3032025-10-03T21:15:10.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3042025-10-03T21:15:10.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3052025-10-03T21:15:10.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3062025-10-03T21:15:10.340ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 150.535169ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
3072025-10-03T21:15:10.340ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
3082025-10-03T21:15:10.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
3092025-10-03T21:15:10.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3102025-10-03T21:15:10.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3112025-10-03T21:15:10.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3122025-10-03T21:15:10.342ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3132025-10-03T21:15:10.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3142025-10-03T21:15:10.344ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
3152025-10-03T21:15:10.344ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = SledList step id = Fetch
3162025-10-03T21:15:10.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3172025-10-03T21:15:10.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3182025-10-03T21:15:10.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3192025-10-03T21:15:10.345ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
3202025-10-03T21:15:10.346ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 156.7435ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
3212025-10-03T21:15:10.346ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3222025-10-03T21:15:10.346ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
3232025-10-03T21:15:10.346ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 157.383336ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3242025-10-03T21:15:10.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3252025-10-03T21:15:10.349ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
3262025-10-03T21:15:10.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3272025-10-03T21:15:10.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3282025-10-03T21:15:10.350ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
3292025-10-03T21:15:10.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3302025-10-03T21:15:10.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3312025-10-03T21:15:10.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3322025-10-03T21:15:10.355ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
3332025-10-03T21:15:10.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3342025-10-03T21:15:10.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3352025-10-03T21:15:10.366ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3362025-10-03T21:15:10.366ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
3372025-10-03T21:15:10.366ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3382025-10-03T21:15:10.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3392025-10-03T21:15:10.368ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3402025-10-03T21:15:10.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3412025-10-03T21:15:10.375ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3422025-10-03T21:15:10.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3432025-10-03T21:15:10.381ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
3442025-10-03T21:15:10.381ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3452025-10-03T21:15:10.381ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-10-03 21:07:14.470425 UTC dns_group = external generation = 2
3462025-10-03T21:15:10.381ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 193.431444ms iteration = 9 status = {"generation":2}
3472025-10-03T21:15:10.387ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
3482025-10-03T21:15:10.387ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3492025-10-03T21:15:10.388ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f1f1b3b-d9ec-4e43-86ee-51894ac53d4a", "content-length": "26", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} })
3502025-10-03T21:15:10.388ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3512025-10-03T21:15:10.388ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
3522025-10-03T21:15:10.388ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3532025-10-03T21:15:10.389ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5e8b707-b6f6-4328-918a-97616d55c2af", "content-length": "26", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} })
3542025-10-03T21:15:10.389ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3552025-10-03T21:15:10.391ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3562025-10-03T21:15:10.391ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
3572025-10-03T21:15:10.392ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3582025-10-03T21:15:10.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3592025-10-03T21:15:10.397ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 208.22545ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
3602025-10-03T21:15:10.408ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3612025-10-03T21:15:10.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3622025-10-03T21:15:10.412ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3632025-10-03T21:15:10.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3642025-10-03T21:15:10.415ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3652025-10-03T21:15:10.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
3662025-10-03T21:15:10.418ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3672025-10-03T21:15:10.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3682025-10-03T21:15:10.419ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3692025-10-03T21:15:10.421ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 232.09764ms iteration = 17 status = 0
3702025-10-03T21:15:10.450ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3712025-10-03T21:15:10.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3722025-10-03T21:15:10.452ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
3732025-10-03T21:15:10.452ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3742025-10-03T21:15:10.452ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3370390-9e36-468b-8fea-5dec6815b9fb", "content-length": "26", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} })
3752025-10-03T21:15:10.452ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3762025-10-03T21:15:10.458ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3772025-10-03T21:15:10.466ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3782025-10-03T21:15:10.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3792025-10-03T21:15:10.468ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3802025-10-03T21:15:10.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3812025-10-03T21:15:10.468ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3822025-10-03T21:15:10.471ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3832025-10-03T21:15:10.471ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3842025-10-03T21:15:10.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3852025-10-03T21:15:10.474ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3862025-10-03T21:15:10.477ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3872025-10-03T21:15:10.477ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3882025-10-03T21:15:10.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3892025-10-03T21:15:10.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3902025-10-03T21:15:10.488ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3912025-10-03T21:15:10.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3922025-10-03T21:15:10.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3932025-10-03T21:15:10.496ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
3942025-10-03T21:15:10.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3952025-10-03T21:15:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3962025-10-03T21:15:10.497ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 307.39753ms iteration = 26 status = {"physical_disks_added":0}
3972025-10-03T21:15:10.501ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = DeployNexusRecords step id = Ensure
3982025-10-03T21:15:10.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3992025-10-03T21:15:10.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
4002025-10-03T21:15:10.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
4012025-10-03T21:15:10.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4022025-10-03T21:15:10.503ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
4032025-10-03T21:15:10.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4042025-10-03T21:15:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4052025-10-03T21:15:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
4062025-10-03T21:15:10.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
4072025-10-03T21:15:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
4082025-10-03T21:15:10.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
4092025-10-03T21:15:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
4102025-10-03T21:15:10.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4112025-10-03T21:15:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
4122025-10-03T21:15:10.554ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4132025-10-03T21:15:10.554ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4142025-10-03T21:15:10.554ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6e7949e-0914-4f76-a290-0d543606d086", "content-length": "372", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} })
4152025-10-03T21:15:10.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4162025-10-03T21:15:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4172025-10-03T21:15:10.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4182025-10-03T21:15:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4192025-10-03T21:15:10.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4202025-10-03T21:15:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4212025-10-03T21:15:10.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4222025-10-03T21:15:10.576ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
4232025-10-03T21:15:10.576ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4242025-10-03T21:15:10.576ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d1667b83-fdae-4e1a-9144-ed4844bb4063", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} })
4252025-10-03T21:15:10.576ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4262025-10-03T21:15:10.577ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = SledAgent step id = Ensure
4272025-10-03T21:15:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4282025-10-03T21:15:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4292025-10-03T21:15:10.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4302025-10-03T21:15:10.579ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4312025-10-03T21:15:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4322025-10-03T21:15:10.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4332025-10-03T21:15:10.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4342025-10-03T21:15:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4352025-10-03T21:15:10.584ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4362025-10-03T21:15:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4372025-10-03T21:15:10.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4382025-10-03T21:15:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 398.290086ms iteration = 17 status = {}
4392025-10-03T21:15:10.587ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4402025-10-03T21:15:10.587ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4412025-10-03T21:15:10.588ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5deb82ff-05bb-413d-88d6-19e277c3c6dc", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} })
4422025-10-03T21:15:10.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
4432025-10-03T21:15:10.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4442025-10-03T21:15:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4452025-10-03T21:15:10.590ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 400.744653ms iteration = 17 status = {}
4462025-10-03T21:15:10.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4472025-10-03T21:15:10.596ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4482025-10-03T21:15:10.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4492025-10-03T21:15:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4502025-10-03T21:15:10.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4512025-10-03T21:15:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4522025-10-03T21:15:10.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4532025-10-03T21:15:10.603ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
4542025-10-03T21:15:10.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4552025-10-03T21:15:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4562025-10-03T21:15:10.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4572025-10-03T21:15:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4582025-10-03T21:15:10.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4592025-10-03T21:15:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4602025-10-03T21:15:10.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4612025-10-03T21:15:10.613ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4622025-10-03T21:15:10.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4632025-10-03T21:15:10.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4642025-10-03T21:15:10.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4652025-10-03T21:15:10.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4662025-10-03T21:15:10.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4672025-10-03T21:15:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4682025-10-03T21:15:10.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4692025-10-03T21:15:10.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4702025-10-03T21:15:10.624ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
4712025-10-03T21:15:10.624ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 434.587465ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
4722025-10-03T21:15:10.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4732025-10-03T21:15:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4742025-10-03T21:15:10.628ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
4752025-10-03T21:15:10.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4762025-10-03T21:15:10.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4772025-10-03T21:15:10.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4782025-10-03T21:15:10.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4792025-10-03T21:15:10.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4802025-10-03T21:15:10.636ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4812025-10-03T21:15:10.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4822025-10-03T21:15:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4832025-10-03T21:15:10.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4842025-10-03T21:15:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4852025-10-03T21:15:10.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4862025-10-03T21:15:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4872025-10-03T21:15:10.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4882025-10-03T21:15:10.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
4892025-10-03T21:15:10.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4902025-10-03T21:15:10.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4912025-10-03T21:15:10.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4922025-10-03T21:15:10.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4932025-10-03T21:15:10.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4942025-10-03T21:15:10.660ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
4952025-10-03T21:15:10.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4962025-10-03T21:15:10.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4972025-10-03T21:15:10.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4982025-10-03T21:15:10.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4992025-10-03T21:15:10.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5002025-10-03T21:15:10.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
5012025-10-03T21:15:10.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5022025-10-03T21:15:10.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5032025-10-03T21:15:10.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5042025-10-03T21:15:10.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5052025-10-03T21:15:10.675ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
5062025-10-03T21:15:10.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5072025-10-03T21:15:10.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
5082025-10-03T21:15:10.677ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 489.320353ms iteration = 17 status = null
5092025-10-03T21:15:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
5102025-10-03T21:15:10.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5112025-10-03T21:15:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5122025-10-03T21:15:10.680ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5132025-10-03T21:15:10.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5142025-10-03T21:15:10.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5152025-10-03T21:15:10.681ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0000c09a-32b2-451e-b377-777a5fee94d9", "content-length": "691", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} })
5162025-10-03T21:15:10.681ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
5172025-10-03T21:15:10.681ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
5182025-10-03T21:15:10.681ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
5192025-10-03T21:15:10.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5202025-10-03T21:15:10.683ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
5212025-10-03T21:15:10.683ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "1ad5183d-daf7-4927-b5b7-0f3e2f782cb9", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5222025-10-03T21:15:10.683ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5232025-10-03T21:15:10.684ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 494.152797ms iteration = 17 status = {}
5242025-10-03T21:15:10.685ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "4f91b375-bdf5-43ec-b599-910bda0db5ec", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5252025-10-03T21:15:10.685ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5262025-10-03T21:15:10.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
5272025-10-03T21:15:10.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5282025-10-03T21:15:10.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5292025-10-03T21:15:10.686ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5302025-10-03T21:15:10.686ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 498.576407ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
5312025-10-03T21:15:10.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5322025-10-03T21:15:10.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5332025-10-03T21:15:10.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5342025-10-03T21:15:10.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
5352025-10-03T21:15:10.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5362025-10-03T21:15:10.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5372025-10-03T21:15:10.729ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
5382025-10-03T21:15:10.729ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
5392025-10-03T21:15:10.729ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = cfdcfb32-4a46-4f79-9af1-ece8a39fb361 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
5402025-10-03T21:15:10.729ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
5412025-10-03T21:15:10.729ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 540.330929ms iteration = 20 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
5422025-10-03T21:15:10.754ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5432025-10-03T21:15:10.755ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "447dcf25-0f75-48fb-9402-b833f7ef9c00", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
5442025-10-03T21:15:10.755ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
5452025-10-03T21:15:10.757ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "e4adede5-166d-475e-b36a-542112e9ffab", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
5462025-10-03T21:15:10.757ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
5472025-10-03T21:15:10.759ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d86eaa40-6911-419c-b183-70fb0206ce44", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
5482025-10-03T21:15:10.759ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
5492025-10-03T21:15:10.761ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
5502025-10-03T21:15:10.761ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 573.098919ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5512025-10-03T21:15:10.782ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
5522025-10-03T21:15:10.784ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "2352aaae-2a77-4afe-bb33-36a8490ffc20", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} }) sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
5532025-10-03T21:15:10.785ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = FirewallRules step id = Ensure
5542025-10-03T21:15:10.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5552025-10-03T21:15:10.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5562025-10-03T21:15:10.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5572025-10-03T21:15:10.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5582025-10-03T21:15:10.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5592025-10-03T21:15:10.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5602025-10-03T21:15:10.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5612025-10-03T21:15:10.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5622025-10-03T21:15:10.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5632025-10-03T21:15:10.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5642025-10-03T21:15:10.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5652025-10-03T21:15:10.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5662025-10-03T21:15:10.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5672025-10-03T21:15:10.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5682025-10-03T21:15:10.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5692025-10-03T21:15:10.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5702025-10-03T21:15:10.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5712025-10-03T21:15:10.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5722025-10-03T21:15:10.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5732025-10-03T21:15:10.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5742025-10-03T21:15:10.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5752025-10-03T21:15:10.808ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5762025-10-03T21:15:10.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5772025-10-03T21:15:10.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5782025-10-03T21:15:10.811ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5792025-10-03T21:15:10.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5802025-10-03T21:15:10.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5812025-10-03T21:15:10.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5822025-10-03T21:15:10.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5832025-10-03T21:15:10.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5842025-10-03T21:15:10.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5852025-10-03T21:15:10.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5862025-10-03T21:15:10.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5872025-10-03T21:15:10.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5882025-10-03T21:15:10.818ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5892025-10-03T21:15:10.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5902025-10-03T21:15:10.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5912025-10-03T21:15:10.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5922025-10-03T21:15:10.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5932025-10-03T21:15:10.821ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
5942025-10-03T21:15:10.821ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5952025-10-03T21:15:10.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5962025-10-03T21:15:10.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5972025-10-03T21:15:10.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5982025-10-03T21:15:10.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5992025-10-03T21:15:10.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6002025-10-03T21:15:10.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6012025-10-03T21:15:10.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6022025-10-03T21:15:10.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6032025-10-03T21:15:10.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6042025-10-03T21:15:10.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6052025-10-03T21:15:10.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6062025-10-03T21:15:10.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6072025-10-03T21:15:10.833ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
6082025-10-03T21:15:10.833ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
6092025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
6102025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
6112025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6122025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6132025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6142025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6152025-10-03T21:15:10.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6162025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
6172025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
6182025-10-03T21:15:10.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6192025-10-03T21:15:10.834ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
6202025-10-03T21:15:10.834ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6212025-10-03T21:15:10.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6222025-10-03T21:15:10.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6232025-10-03T21:15:10.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6242025-10-03T21:15:10.836ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
6252025-10-03T21:15:10.836ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6262025-10-03T21:15:10.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6272025-10-03T21:15:10.842ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
6282025-10-03T21:15:10.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6292025-10-03T21:15:10.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6302025-10-03T21:15:10.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6312025-10-03T21:15:10.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6322025-10-03T21:15:10.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6332025-10-03T21:15:10.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6342025-10-03T21:15:10.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6352025-10-03T21:15:10.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
6362025-10-03T21:15:10.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6372025-10-03T21:15:10.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6382025-10-03T21:15:10.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6392025-10-03T21:15:10.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6402025-10-03T21:15:10.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6412025-10-03T21:15:10.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6422025-10-03T21:15:10.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6432025-10-03T21:15:10.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6442025-10-03T21:15:10.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6452025-10-03T21:15:10.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6462025-10-03T21:15:10.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6472025-10-03T21:15:10.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6482025-10-03T21:15:10.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6492025-10-03T21:15:10.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6502025-10-03T21:15:10.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6512025-10-03T21:15:10.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6522025-10-03T21:15:10.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6532025-10-03T21:15:10.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
6542025-10-03T21:15:10.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6552025-10-03T21:15:10.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6562025-10-03T21:15:10.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6572025-10-03T21:15:10.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6582025-10-03T21:15:10.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6592025-10-03T21:15:10.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6602025-10-03T21:15:10.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6612025-10-03T21:15:10.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
6622025-10-03T21:15:10.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6632025-10-03T21:15:10.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6642025-10-03T21:15:10.890ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
6652025-10-03T21:15:10.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6662025-10-03T21:15:10.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6672025-10-03T21:15:10.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6682025-10-03T21:15:10.893ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
6692025-10-03T21:15:10.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6702025-10-03T21:15:10.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6712025-10-03T21:15:10.895ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
6722025-10-03T21:15:10.895ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
6732025-10-03T21:15:10.895ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
6742025-10-03T21:15:10.953ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), time_created: 2025-10-03T21:07:05.994Z, time_modified: 2025-10-03T21:07:05.994Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
6752025-10-03T21:15:10.953ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
6762025-10-03T21:15:10.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6772025-10-03T21:15:10.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6782025-10-03T21:15:10.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6792025-10-03T21:15:10.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6802025-10-03T21:15:10.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6812025-10-03T21:15:10.992ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
6822025-10-03T21:15:10.992ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
6832025-10-03T21:15:10.999ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "8004408a-6234-4a46-b512-da5339d3306c", "date": "Fri, 03 Oct 2025 21:15:10 GMT"} })
6842025-10-03T21:15:11.001ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
6852025-10-03T21:15:11.001ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = Dns step id = Ensure
6862025-10-03T21:15:11.001ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
6872025-10-03T21:15:11.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6882025-10-03T21:15:11.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6892025-10-03T21:15:11.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6902025-10-03T21:15:11.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6912025-10-03T21:15:11.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6922025-10-03T21:15:11.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
6932025-10-03T21:15:11.021ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "01f2646c-ba6c-4a4d-8beb-592f73e93e16", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
6942025-10-03T21:15:11.021ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
6952025-10-03T21:15:11.021ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6962025-10-03T21:15:11.027ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "37a88d9c-719d-4bfe-b9dc-5b75ab0d6db1", "content-length": "682", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
6972025-10-03T21:15:11.027ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6982025-10-03T21:15:11.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6992025-10-03T21:15:11.031ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7002025-10-03T21:15:11.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7012025-10-03T21:15:11.035ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7022025-10-03T21:15:11.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7032025-10-03T21:15:11.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7042025-10-03T21:15:11.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7052025-10-03T21:15:11.041ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
7062025-10-03T21:15:11.041ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
7072025-10-03T21:15:11.041ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
7082025-10-03T21:15:11.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7092025-10-03T21:15:11.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7102025-10-03T21:15:11.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7112025-10-03T21:15:11.044ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 855.069461ms iteration = 17 status = {}
7122025-10-03T21:15:11.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
7132025-10-03T21:15:11.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7142025-10-03T21:15:11.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7152025-10-03T21:15:11.052ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
7162025-10-03T21:15:11.052ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
7172025-10-03T21:15:11.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7182025-10-03T21:15:11.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7192025-10-03T21:15:11.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7202025-10-03T21:15:11.056ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
7212025-10-03T21:15:11.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7222025-10-03T21:15:11.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7232025-10-03T21:15:11.060ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
7242025-10-03T21:15:11.060ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
7252025-10-03T21:15:11.060ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
7262025-10-03T21:15:11.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7272025-10-03T21:15:11.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7282025-10-03T21:15:11.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7292025-10-03T21:15:11.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7302025-10-03T21:15:11.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7312025-10-03T21:15:11.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
7322025-10-03T21:15:11.066ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
7332025-10-03T21:15:11.066ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
7342025-10-03T21:15:11.066ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
7352025-10-03T21:15:11.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7362025-10-03T21:15:11.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7372025-10-03T21:15:11.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7382025-10-03T21:15:11.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
7392025-10-03T21:15:11.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7402025-10-03T21:15:11.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7412025-10-03T21:15:11.112ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
7422025-10-03T21:15:11.112ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
7432025-10-03T21:15:11.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7442025-10-03T21:15:11.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7452025-10-03T21:15:11.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462025-10-03T21:15:11.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
7472025-10-03T21:15:11.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7482025-10-03T21:15:11.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7492025-10-03T21:15:11.119ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
7502025-10-03T21:15:11.119ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
7512025-10-03T21:15:11.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7522025-10-03T21:15:11.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7532025-10-03T21:15:11.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7542025-10-03T21:15:11.122ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
7552025-10-03T21:15:11.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7562025-10-03T21:15:11.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7572025-10-03T21:15:11.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7582025-10-03T21:15:11.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7592025-10-03T21:15:11.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7602025-10-03T21:15:11.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7612025-10-03T21:15:11.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7622025-10-03T21:15:11.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
7632025-10-03T21:15:11.132ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
7642025-10-03T21:15:11.132ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
7652025-10-03T21:15:11.132ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = OmicronZones step id = Cleanup
7662025-10-03T21:15:11.132ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = OmicronZones step id = Cleanup
7672025-10-03T21:15:11.132ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = PhysicalDisks step id = Cleanup
7682025-10-03T21:15:11.132ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = Clickhouse step id = Ensure
7692025-10-03T21:15:11.132ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = Clickhouse step id = Ensure
7702025-10-03T21:15:11.168ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
7712025-10-03T21:15:11.172ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "76964081-b9a4-43a3-aebb-2c8a5cb97334", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
7722025-10-03T21:15:11.175ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = SupportBundles step id = Cleanup
7732025-10-03T21:15:11.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7742025-10-03T21:15:11.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7752025-10-03T21:15:11.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7762025-10-03T21:15:11.178ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
7772025-10-03T21:15:11.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7782025-10-03T21:15:11.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7792025-10-03T21:15:11.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7802025-10-03T21:15:11.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7812025-10-03T21:15:11.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7822025-10-03T21:15:11.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
7832025-10-03T21:15:11.222ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 457 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 0dff1329-d29e-4e0c-aa9e-e8a5f7a6f807 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
7842025-10-03T21:15:11.226ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = OmicronZones step id = Cleanup
7852025-10-03T21:15:11.226ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
7862025-10-03T21:15:11.226ZDEBGnexus (BackgroundTasks): re-assign sagas: found Nexus instances background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
7872025-10-03T21:15:11.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7882025-10-03T21:15:11.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7892025-10-03T21:15:11.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7902025-10-03T21:15:11.230ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
7912025-10-03T21:15:11.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7922025-10-03T21:15:11.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7932025-10-03T21:15:11.264ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:54 nexus_zone_ids = []
7942025-10-03T21:15:11.264ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(675bea75-8e2d-4986-86f2-b590a30bb084 (blueprint), false)
7952025-10-03T21:15:11.264ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = Cockroach step id = Ensure
7962025-10-03T21:15:11.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7972025-10-03T21:15:11.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7982025-10-03T21:15:11.278ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
7992025-10-03T21:15:11.278ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 63aac8ac-3097-4d44-a07d-618d53440f8e step component = MgsUpdates step id = Ensure
8002025-10-03T21:15:11.278ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Dependency
8012025-10-03T21:15:11.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 10 reason = Dependency
8022025-10-03T21:15:11.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8032025-10-03T21:15:11.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8042025-10-03T21:15:11.278ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.089961944s iteration = 10 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4110,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":155991831,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":155991781,"secs":0}},"event_index":1,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":156060192,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":156244854,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":156244804,"secs":0}},"event_index":2,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":312321816,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":76154819,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":76154769,"secs":0}},"event_index":3,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":388497875,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":208658806,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":208658776,"secs":0}},"event_index":4,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":597173901,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":216043339,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":216043299,"secs":0}},"event_index":5,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":813229580,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":130862217,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":130862177,"secs":0}},"event_index":6,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":944100727,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4120,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4080,"secs":0}},"event_index":7,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":944118227,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2260,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2220,"secs":0}},"event_index":8,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":944122767,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2680,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2640,"secs":0}},"event_index":9,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":944126797,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1870,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1830,"secs":0}},"event_index":10,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":944130897,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":42229617,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":42229567,"secs":0}},"event_index":11,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":986361584,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":51305219,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":51305179,"secs":0}},"event_index":12,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":37678913,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":37748217,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":37748177,"secs":0}},"event_index":13,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":75440830,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":14163090,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":14163050,"secs":0}},"event_index":14,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":89626220,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5450,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5410,"secs":0}},"event_index":15,"execution_id":"68dab6cb-509d-4123-8ec1-740b90dfe71d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":89642930,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"675bea75-8e2d-4986-86f2-b590a30bb084"}
8052025-10-03T21:15:11.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8062025-10-03T21:15:11.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
8072025-10-03T21:15:11.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8082025-10-03T21:15:11.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8092025-10-03T21:15:11.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
8102025-10-03T21:15:11.283ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8112025-10-03T21:15:11.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8122025-10-03T21:15:11.283ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8132025-10-03T21:15:11.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8142025-10-03T21:15:11.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
8152025-10-03T21:15:11.285ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8162025-10-03T21:15:11.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8172025-10-03T21:15:11.289ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8182025-10-03T21:15:11.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
8192025-10-03T21:15:11.291ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8202025-10-03T21:15:11.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8212025-10-03T21:15:11.293ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
8222025-10-03T21:15:11.293ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 15.02515ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
8232025-10-03T21:15:11.406ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(37e8ce34-7581-447e-b472-ae3b133a1905, 0), (f045050f-9d07-4886-92bb-95ce48733e72, 0), (ff7faa4e-ee9d-427a-924f-4d8aad298aef, 0), (d9bf6af8-0c02-4ab9-80a0-fdc64bfdcdbd, 0)] collection_id = 37e8ce34-7581-447e-b472-ae3b133a1905
8242025-10-03T21:15:11.406ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8252025-10-03T21:15:11.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
8262025-10-03T21:15:11.605ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 37e8ce34-7581-447e-b472-ae3b133a1905 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
8272025-10-03T21:15:11.605ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8282025-10-03T21:15:11.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
8292025-10-03T21:15:11.609ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(f045050f-9d07-4886-92bb-95ce48733e72, 0), (ff7faa4e-ee9d-427a-924f-4d8aad298aef, 0), (d9bf6af8-0c02-4ab9-80a0-fdc64bfdcdbd, 0)]
8302025-10-03T21:15:11.610ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 4957, tv_nsec: 100751123 } })
8312025-10-03T21:15:11.642ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("26cdd4d2-d0ac-43df-bf3e-350fd160efec.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41cf4311-4781-41d3-b8bc-181c344891b0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b55c423c-6cde-4185-879d-3a1c4cf01de4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b807e991-1a36-4877-8937-3efb3994dbc2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::3)) }], valid_until: Instant { tv_sec: 4957, tv_nsec: 133651234 } })
8322025-10-03T21:15:11.643ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
8332025-10-03T21:15:11.643ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
8342025-10-03T21:15:11.643ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
8352025-10-03T21:15:11.644ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0991166b-633a-4b31-914b-48ce01ff2a47", "content-length": "476", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8362025-10-03T21:15:11.644ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
8372025-10-03T21:15:11.644ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0cdd33a-a7bf-4479-8731-524c778477eb", "content-length": "734", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8382025-10-03T21:15:11.644ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
8392025-10-03T21:15:11.644ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "39ef994b-b931-493d-96a4-b45da911103f", "content-length": "111", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8402025-10-03T21:15:11.644ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
8412025-10-03T21:15:11.645ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e74159e-3c0e-4d97-9eae-8598e56609b2", "content-length": "111", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8422025-10-03T21:15:11.645ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
8432025-10-03T21:15:11.645ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bb90bca-4841-4677-b159-a99fd07be363", "content-length": "163", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8442025-10-03T21:15:11.645ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
8452025-10-03T21:15:11.646ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "04f15b97-d602-4385-8bd8-51d7e1559ae9", "content-length": "163", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8462025-10-03T21:15:11.646ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
8472025-10-03T21:15:11.646ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bd6cdd1-623e-41c9-81c4-4a4135065b46", "content-length": "166", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8482025-10-03T21:15:11.646ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
8492025-10-03T21:15:11.647ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c870615-c262-4b04-9979-66428e7722af", "content-length": "166", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8502025-10-03T21:15:11.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
8512025-10-03T21:15:11.647ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff796aaf-732f-472c-8c87-77999b5ee29e", "content-length": "702", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8522025-10-03T21:15:11.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8532025-10-03T21:15:11.647ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d523ac6e-1649-4562-9cf7-892da972cac0", "content-length": "727", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8542025-10-03T21:15:11.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8552025-10-03T21:15:11.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "28cdbfd0-53a3-4a15-8bde-66331adb6b70", "content-length": "729", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8562025-10-03T21:15:11.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
8572025-10-03T21:15:11.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7a2f2d2-9eee-429f-8e7f-9c8c1ac919c4", "content-length": "728", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8582025-10-03T21:15:11.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
8592025-10-03T21:15:11.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dddea838-1d01-41fa-b8cd-016d8f48d1ec", "content-length": "722", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8602025-10-03T21:15:11.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
8612025-10-03T21:15:11.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd56ba65-908b-4e9b-ba34-a7077d7d394a", "content-length": "10", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8622025-10-03T21:15:11.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
8632025-10-03T21:15:11.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "283afa8a-df51-4aab-b2d4-3ef4dd729bb3", "content-length": "144", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8642025-10-03T21:15:11.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
8652025-10-03T21:15:11.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b32aa64a-1012-4178-92db-79a5a8164a39", "content-length": "147", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8662025-10-03T21:15:11.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
8672025-10-03T21:15:11.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "08abc724-e2b2-464c-be98-7a1f29efe355", "content-length": "109", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8682025-10-03T21:15:11.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
8692025-10-03T21:15:11.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55438b9c-e631-4981-91aa-6c0a922febee", "content-length": "109", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8702025-10-03T21:15:11.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
8712025-10-03T21:15:11.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d649d88d-8671-454f-8094-b3507c83ba2c", "content-length": "163", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8722025-10-03T21:15:11.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
8732025-10-03T21:15:11.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b43432c2-6adb-48e6-abb4-fae90cf50ee8", "content-length": "163", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8742025-10-03T21:15:11.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
8752025-10-03T21:15:11.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b38ecdcb-e48f-4538-a4f2-42490d00d981", "content-length": "166", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8762025-10-03T21:15:11.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
8772025-10-03T21:15:11.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ab6fc2e-8308-4ba8-ada6-51086f0d0e80", "content-length": "166", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8782025-10-03T21:15:11.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
8792025-10-03T21:15:11.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "22ad6453-8358-4cae-99a3-790ccc3bc238", "content-length": "702", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8802025-10-03T21:15:11.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8812025-10-03T21:15:11.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "939d92f4-10d6-4354-9db0-bd0da0f5f2dd", "content-length": "727", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8822025-10-03T21:15:11.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8832025-10-03T21:15:11.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c6d3f64-f158-426c-acf7-0843f1678d01", "content-length": "729", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8842025-10-03T21:15:11.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
8852025-10-03T21:15:11.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7bd0903-3c6e-41b8-a77f-59f896fb889a", "content-length": "728", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8862025-10-03T21:15:11.654ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8872025-10-03T21:15:11.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
8882025-10-03T21:15:11.656ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8892025-10-03T21:15:11.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
8902025-10-03T21:15:11.658ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8912025-10-03T21:15:11.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
8922025-10-03T21:15:11.661ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8932025-10-03T21:15:11.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
8942025-10-03T21:15:11.663ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8952025-10-03T21:15:11.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
8962025-10-03T21:15:11.698ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
8972025-10-03T21:15:11.698ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
8982025-10-03T21:15:11.764ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "326a792f-bbcf-4888-b3b1-810983dbe4b3", "content-length": "88632", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
8992025-10-03T21:15:11.766ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
9002025-10-03T21:15:11.766ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
9012025-10-03T21:15:11.766ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
9022025-10-03T21:15:11.766ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
9032025-10-03T21:15:11.766ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
9042025-10-03T21:15:11.766ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
9052025-10-03T21:15:11.766ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
9062025-10-03T21:15:11.766ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
9072025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff55904a-394e-4530-85e7-108f9782b107", "content-length": "64", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9082025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
9092025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e33eeb8-47c3-4d08-8d62-c55e651d7aab", "content-length": "64", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9102025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "657cb9ac-fbaf-4cc0-8804-fd6346f3224d", "content-length": "64", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9112025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
9122025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9e08df2-eb02-4e18-b8b7-98bae853f754", "content-length": "64", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9132025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5616b43b-81ac-442a-b25d-fc3fdd78e2e4", "content-length": "64", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9142025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
9152025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
9162025-10-03T21:15:11.767ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
9172025-10-03T21:15:11.804ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f81fa747-ed00-46b7-b47b-e1acfca2c74b", "content-length": "400473", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9182025-10-03T21:15:11.805ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1731fa31-616f-42e7-8708-4b4d042a3cb2", "content-length": "400195", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9192025-10-03T21:15:11.805ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6af73cf-f04b-4c8e-b2dc-014b4afb0158", "content-length": "392385", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9202025-10-03T21:15:11.805ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2187f4b-88ed-4169-8940-8ce711e160b3", "content-length": "395720", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9212025-10-03T21:15:11.809ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee3366e3-f51d-4ce5-b265-c48b0eb0cb71", "content-length": "427170", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9222025-10-03T21:15:11.811ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
9232025-10-03T21:15:11.811ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
9242025-10-03T21:15:11.811ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
9252025-10-03T21:15:11.811ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
9262025-10-03T21:15:11.811ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
9272025-10-03T21:15:11.841ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 (service)
9282025-10-03T21:15:11.841ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
9292025-10-03T21:15:11.844ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b49acef-7142-4ed9-98d5-d2c2f1e94149", "content-length": "124", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9302025-10-03T21:15:11.846ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
9312025-10-03T21:15:11.937ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
9322025-10-03T21:15:11.937ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
9332025-10-03T21:15:11.937ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0414f78-1cdf-4db4-b84c-fe69858182ba", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9342025-10-03T21:15:11.937ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
9352025-10-03T21:15:11.939ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
9362025-10-03T21:15:11.939ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
9372025-10-03T21:15:11.939ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7a7f428-c5a8-4d66-aebf-5ea03e0261ea", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9382025-10-03T21:15:11.940ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
9392025-10-03T21:15:11.940ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
9402025-10-03T21:15:11.940ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
9412025-10-03T21:15:11.941ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "893ffcdc-c43d-410f-b662-25f2b4f5b70d", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:15:11 GMT"} })
9422025-10-03T21:15:11.941ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
9432025-10-03T21:15:11.943ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
9442025-10-03T21:15:11.943ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
9452025-10-03T21:15:11.944ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9462025-10-03T21:15:11.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9472025-10-03T21:15:11.947ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9482025-10-03T21:15:11.949ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
9492025-10-03T21:15:11.949ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9502025-10-03T21:15:11.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9512025-10-03T21:15:12.164ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = e33257de-ae23-49ef-9065-e527da3bdcfa file = nexus/db-queries/src/db/datastore/inventory.rs:1614
9522025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = e33257de-ae23-49ef-9065-e527da3bdcfa time_started = 2025-10-03 21:15:11.643586 UTC
9532025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 21 reason = Dependency
9542025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Dependency
9552025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
9562025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 12 reason = Dependency
9572025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9582025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
9592025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 885.990741ms iteration = 10 status = {"collection_id":"e33257de-ae23-49ef-9065-e527da3bdcfa","time_done":"2025-10-03 21:15:11.943055 UTC","time_started":"2025-10-03 21:15:11.643586 UTC"}
9602025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9612025-10-03T21:15:12.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
9622025-10-03T21:15:12.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
9632025-10-03T21:15:12.166ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9642025-10-03T21:15:12.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9652025-10-03T21:15:12.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9662025-10-03T21:15:12.169ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9672025-10-03T21:15:12.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9682025-10-03T21:15:12.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
9692025-10-03T21:15:12.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
9702025-10-03T21:15:12.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
9712025-10-03T21:15:12.171ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9722025-10-03T21:15:12.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9732025-10-03T21:15:12.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
9742025-10-03T21:15:12.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
9752025-10-03T21:15:12.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9762025-10-03T21:15:12.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9772025-10-03T21:15:12.174ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9782025-10-03T21:15:12.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9792025-10-03T21:15:12.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
9802025-10-03T21:15:12.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
9812025-10-03T21:15:12.222ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 57.785743ms iteration = 27 status = {"physical_disks_added":0}
9822025-10-03T21:15:12.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9832025-10-03T21:15:12.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
9842025-10-03T21:15:12.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
9852025-10-03T21:15:12.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
9862025-10-03T21:15:12.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
9872025-10-03T21:15:12.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
9882025-10-03T21:15:12.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9892025-10-03T21:15:12.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9902025-10-03T21:15:12.228ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
9912025-10-03T21:15:12.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
9922025-10-03T21:15:12.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
9932025-10-03T21:15:12.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
9942025-10-03T21:15:12.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
9952025-10-03T21:15:12.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
9962025-10-03T21:15:12.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
9972025-10-03T21:15:12.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
9982025-10-03T21:15:12.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9992025-10-03T21:15:12.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10002025-10-03T21:15:12.233ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
10012025-10-03T21:15:12.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10022025-10-03T21:15:12.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10032025-10-03T21:15:12.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
10042025-10-03T21:15:12.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
10052025-10-03T21:15:12.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
10062025-10-03T21:15:12.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10072025-10-03T21:15:12.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10082025-10-03T21:15:12.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10092025-10-03T21:15:12.239ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
10102025-10-03T21:15:12.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
10112025-10-03T21:15:12.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
10122025-10-03T21:15:12.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10132025-10-03T21:15:12.242ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
10142025-10-03T21:15:12.242ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 78.446466ms iteration = 12 status = {"blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","inventory_collection_id":"e33257de-ae23-49ef-9065-e527da3bdcfa","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
10152025-10-03T21:15:12.243ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
10162025-10-03T21:15:12.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10172025-10-03T21:15:12.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10182025-10-03T21:15:12.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10192025-10-03T21:15:12.246ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
10202025-10-03T21:15:12.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10212025-10-03T21:15:12.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10222025-10-03T21:15:12.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10232025-10-03T21:15:12.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10242025-10-03T21:15:12.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10252025-10-03T21:15:12.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10262025-10-03T21:15:12.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10272025-10-03T21:15:12.256ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
10282025-10-03T21:15:12.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10292025-10-03T21:15:12.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10302025-10-03T21:15:12.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10312025-10-03T21:15:12.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10322025-10-03T21:15:12.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10332025-10-03T21:15:12.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
10342025-10-03T21:15:12.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10352025-10-03T21:15:12.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10362025-10-03T21:15:12.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10372025-10-03T21:15:12.266ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
10382025-10-03T21:15:12.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10392025-10-03T21:15:12.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10402025-10-03T21:15:12.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10412025-10-03T21:15:12.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10422025-10-03T21:15:12.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10432025-10-03T21:15:12.273ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
10442025-10-03T21:15:12.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10452025-10-03T21:15:12.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10462025-10-03T21:15:12.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10472025-10-03T21:15:12.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10482025-10-03T21:15:12.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10492025-10-03T21:15:12.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10502025-10-03T21:15:12.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10512025-10-03T21:15:12.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
10522025-10-03T21:15:12.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10532025-10-03T21:15:12.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10542025-10-03T21:15:12.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10552025-10-03T21:15:12.284ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
10562025-10-03T21:15:12.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10572025-10-03T21:15:12.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10582025-10-03T21:15:12.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10592025-10-03T21:15:12.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10602025-10-03T21:15:12.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10612025-10-03T21:15:12.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10622025-10-03T21:15:12.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10632025-10-03T21:15:12.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
10642025-10-03T21:15:12.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10652025-10-03T21:15:12.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10662025-10-03T21:15:12.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10672025-10-03T21:15:12.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
10682025-10-03T21:15:12.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10692025-10-03T21:15:12.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10702025-10-03T21:15:12.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10712025-10-03T21:15:12.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10722025-10-03T21:15:12.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10732025-10-03T21:15:12.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10742025-10-03T21:15:12.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10752025-10-03T21:15:12.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
10762025-10-03T21:15:12.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10772025-10-03T21:15:12.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10782025-10-03T21:15:12.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10792025-10-03T21:15:12.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10802025-10-03T21:15:12.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10812025-10-03T21:15:12.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
10822025-10-03T21:15:12.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10832025-10-03T21:15:12.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10842025-10-03T21:15:12.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10852025-10-03T21:15:12.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10862025-10-03T21:15:12.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10872025-10-03T21:15:12.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
10882025-10-03T21:15:12.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10892025-10-03T21:15:12.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10902025-10-03T21:15:12.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10912025-10-03T21:15:12.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10922025-10-03T21:15:12.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10932025-10-03T21:15:12.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
10942025-10-03T21:15:12.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10952025-10-03T21:15:12.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10962025-10-03T21:15:12.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10972025-10-03T21:15:12.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10982025-10-03T21:15:12.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10992025-10-03T21:15:12.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
11002025-10-03T21:15:12.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11012025-10-03T21:15:12.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11022025-10-03T21:15:12.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11032025-10-03T21:15:12.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11042025-10-03T21:15:12.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11052025-10-03T21:15:12.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
11062025-10-03T21:15:12.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11072025-10-03T21:15:12.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11082025-10-03T21:15:12.338ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
11092025-10-03T21:15:12.338ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
11102025-10-03T21:15:12.338ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = ac3b3365-928e-4928-b132-6ce543c0cbfc parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
11112025-10-03T21:15:12.338ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
11122025-10-03T21:15:12.338ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 174.398439ms iteration = 21 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
11132025-10-03T21:15:15.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 98 reason = Timeout
11142025-10-03T21:15:15.189ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11152025-10-03T21:15:15.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11162025-10-03T21:15:15.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11172025-10-03T21:15:15.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
11182025-10-03T21:15:15.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11192025-10-03T21:15:15.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11202025-10-03T21:15:15.196ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
11212025-10-03T21:15:15.196ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.864357ms iteration = 98 status = {"config_updated":false}
11222025-10-03T21:15:20.188ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
11232025-10-03T21:15:20.189ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11242025-10-03T21:15:20.189ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
11252025-10-03T21:15:20.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 99 reason = Timeout
11262025-10-03T21:15:20.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11272025-10-03T21:15:20.189ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11282025-10-03T21:15:20.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11292025-10-03T21:15:20.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11302025-10-03T21:15:20.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11312025-10-03T21:15:20.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11322025-10-03T21:15:20.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11332025-10-03T21:15:20.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11342025-10-03T21:15:20.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11352025-10-03T21:15:20.193ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.521741ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11362025-10-03T21:15:20.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
11372025-10-03T21:15:20.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11382025-10-03T21:15:20.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11392025-10-03T21:15:20.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11402025-10-03T21:15:20.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11412025-10-03T21:15:20.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11422025-10-03T21:15:20.196ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
11432025-10-03T21:15:20.196ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.375503ms iteration = 99 status = {"config_updated":false}
11442025-10-03T21:15:20.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11452025-10-03T21:15:20.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11462025-10-03T21:15:20.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11472025-10-03T21:15:20.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
11482025-10-03T21:15:20.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11492025-10-03T21:15:20.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11502025-10-03T21:15:20.214ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
11512025-10-03T21:15:20.214ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.245296ms iteration = 50 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
11522025-10-03T21:15:21.222ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 385 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 4403f0a8-2e3a-43fe-abd0-f55eb3fac466 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
11532025-10-03T21:15:25.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 100 reason = Timeout
11542025-10-03T21:15:25.189ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11552025-10-03T21:15:25.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11562025-10-03T21:15:25.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11572025-10-03T21:15:25.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
11582025-10-03T21:15:25.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11592025-10-03T21:15:25.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11602025-10-03T21:15:25.195ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
11612025-10-03T21:15:25.195ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.264941ms iteration = 100 status = {"config_updated":false}
11622025-10-03T21:15:30.189ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
11632025-10-03T21:15:30.189ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11642025-10-03T21:15:30.189ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
11652025-10-03T21:15:30.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 101 reason = Timeout
11662025-10-03T21:15:30.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11672025-10-03T21:15:30.189ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11682025-10-03T21:15:30.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11692025-10-03T21:15:30.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11702025-10-03T21:15:30.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11712025-10-03T21:15:30.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11722025-10-03T21:15:30.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11732025-10-03T21:15:30.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11742025-10-03T21:15:30.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11752025-10-03T21:15:30.192ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.394438ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11762025-10-03T21:15:30.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
11772025-10-03T21:15:30.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11782025-10-03T21:15:30.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11792025-10-03T21:15:30.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11802025-10-03T21:15:30.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11812025-10-03T21:15:30.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11822025-10-03T21:15:30.196ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
11832025-10-03T21:15:30.196ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.963769ms iteration = 101 status = {"config_updated":false}
11842025-10-03T21:15:30.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11852025-10-03T21:15:30.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11862025-10-03T21:15:30.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11872025-10-03T21:15:30.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
11882025-10-03T21:15:30.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11892025-10-03T21:15:30.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11902025-10-03T21:15:30.212ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
11912025-10-03T21:15:30.213ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.501955ms iteration = 51 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
11922025-10-03T21:15:31.222ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 390 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 1ff8197c-d437-4215-96f8-acb6edc8520e response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
11932025-10-03T21:15:35.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 102 reason = Timeout
11942025-10-03T21:15:35.189ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11952025-10-03T21:15:35.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11962025-10-03T21:15:35.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11972025-10-03T21:15:35.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
11982025-10-03T21:15:35.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11992025-10-03T21:15:35.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
12002025-10-03T21:15:35.196ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
12012025-10-03T21:15:35.196ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.456653ms iteration = 102 status = {"config_updated":false}
12022025-10-03T21:15:36.215ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:49405
12032025-10-03T21:15:36.215ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49405 req_id = b91d1b2a-ca53-4b43-b53c-6f07878d8a34 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
12042025-10-03T21:15:36.215ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49405 req_id = b91d1b2a-ca53-4b43-b53c-6f07878d8a34 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
12052025-10-03T21:15:36.218ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3029 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49405 req_id = b91d1b2a-ca53-4b43-b53c-6f07878d8a34 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending
12062025-10-03T21:15:36.218ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49405 req_id = 672a0510-e042-420d-b072-b3051471d1c4 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
12072025-10-03T21:15:36.218ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49405 req_id = 672a0510-e042-420d-b072-b3051471d1c4 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
12082025-10-03T21:15:36.221ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2562 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:49405 req_id = 672a0510-e042-420d-b072-b3051471d1c4 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D
12092025-10-03T21:15:40.188ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
12102025-10-03T21:15:40.188ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
12112025-10-03T21:15:40.188ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 18 reason = Timeout
12122025-10-03T21:15:40.188ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12132025-10-03T21:15:40.188ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
12142025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
12152025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
12162025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 18 reason = Timeout
12172025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
12182025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 16.481\u{b5}s iteration = 18 status = {"disabled":true,"errors":[],"sps":[]}
12192025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
12202025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12212025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12222025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
12232025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
12242025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12252025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
12262025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
12272025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12282025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
12292025-10-03T21:15:40.189ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
12302025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12312025-10-03T21:15:40.189ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
12322025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
12332025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
12342025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
12352025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12362025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12372025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
12382025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
12392025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
12402025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12412025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 103 reason = Timeout
12422025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
12432025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
12442025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
12452025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12462025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
12472025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
12482025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
12492025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
12502025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
12512025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
12522025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
12532025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12542025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12552025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
12562025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12572025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12582025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12592025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12602025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
12612025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
12622025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
12632025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
12642025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12652025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
12662025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
12672025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
12682025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
12692025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
12702025-10-03T21:15:40.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12712025-10-03T21:15:40.191ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 4985, tv_nsec: 682149590 } })
12722025-10-03T21:15:40.192ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
12732025-10-03T21:15:40.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12742025-10-03T21:15:40.192ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 4985, tv_nsec: 682869528 } })
12752025-10-03T21:15:40.192ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12762025-10-03T21:15:40.193ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
12772025-10-03T21:15:40.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12782025-10-03T21:15:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12792025-10-03T21:15:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12802025-10-03T21:15:40.193ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.388101ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
12812025-10-03T21:15:40.194ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12822025-10-03T21:15:40.194ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12832025-10-03T21:15:40.195ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12842025-10-03T21:15:40.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12852025-10-03T21:15:40.196ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.347602ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
12862025-10-03T21:15:40.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12872025-10-03T21:15:40.197ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12882025-10-03T21:15:40.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12892025-10-03T21:15:40.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12902025-10-03T21:15:40.198ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12912025-10-03T21:15:40.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12922025-10-03T21:15:40.198ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12932025-10-03T21:15:40.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12942025-10-03T21:15:40.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
12952025-10-03T21:15:40.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
12962025-10-03T21:15:40.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12972025-10-03T21:15:40.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
12982025-10-03T21:15:40.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12992025-10-03T21:15:40.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13002025-10-03T21:15:40.200ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 10.319696ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
13012025-10-03T21:15:40.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13022025-10-03T21:15:40.201ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
13032025-10-03T21:15:40.201ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.613251ms iteration = 103 status = {"config_updated":false}
13042025-10-03T21:15:40.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13052025-10-03T21:15:40.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13062025-10-03T21:15:40.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13072025-10-03T21:15:40.206ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 4985, tv_nsec: 696572483 } })
13082025-10-03T21:15:40.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13092025-10-03T21:15:40.206ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
13102025-10-03T21:15:40.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13112025-10-03T21:15:40.210ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13122025-10-03T21:15:40.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13132025-10-03T21:15:40.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13142025-10-03T21:15:40.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13152025-10-03T21:15:40.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
13162025-10-03T21:15:40.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13172025-10-03T21:15:40.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13182025-10-03T21:15:40.251ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
13192025-10-03T21:15:40.251ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 61.362123ms iteration = 52 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
13202025-10-03T21:15:40.260ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13212025-10-03T21:15:40.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13222025-10-03T21:15:40.263ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13232025-10-03T21:15:40.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
13242025-10-03T21:15:40.264ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13252025-10-03T21:15:40.267ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13262025-10-03T21:15:40.267ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13272025-10-03T21:15:40.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13282025-10-03T21:15:40.268ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 78.617828ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
13292025-10-03T21:15:40.270ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13302025-10-03T21:15:40.270ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13312025-10-03T21:15:40.270ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
13322025-10-03T21:15:40.270ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13332025-10-03T21:15:40.270ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13342025-10-03T21:15:40.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13352025-10-03T21:15:40.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
13362025-10-03T21:15:40.270ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 80.774562ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
13372025-10-03T21:15:40.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 82.891446ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13382025-10-03T21:15:40.273ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13392025-10-03T21:15:40.273ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13402025-10-03T21:15:40.273ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
13412025-10-03T21:15:40.273ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13422025-10-03T21:15:40.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
13432025-10-03T21:15:40.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
13442025-10-03T21:15:40.274ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
13452025-10-03T21:15:40.274ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 84.050489ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
13462025-10-03T21:15:40.274ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13472025-10-03T21:15:40.274ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13482025-10-03T21:15:40.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13492025-10-03T21:15:40.275ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.595747ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
13502025-10-03T21:15:40.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13512025-10-03T21:15:40.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13522025-10-03T21:15:40.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
13532025-10-03T21:15:40.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 90.39002ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
13542025-10-03T21:15:40.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13552025-10-03T21:15:40.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13562025-10-03T21:15:40.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13572025-10-03T21:15:40.282ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13582025-10-03T21:15:40.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13592025-10-03T21:15:40.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13602025-10-03T21:15:40.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13612025-10-03T21:15:40.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13622025-10-03T21:15:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
13632025-10-03T21:15:40.285ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13642025-10-03T21:15:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13652025-10-03T21:15:40.292ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 102.192834ms iteration = 28 status = {"physical_disks_added":0}
13662025-10-03T21:15:40.312ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
13672025-10-03T21:15:40.312ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13682025-10-03T21:15:40.312ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "942715a6-98d2-4885-bc0d-61d390df0989", "content-length": "26", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
13692025-10-03T21:15:40.312ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13702025-10-03T21:15:40.314ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13712025-10-03T21:15:40.320ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
13722025-10-03T21:15:40.320ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13732025-10-03T21:15:40.320ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "be38c3be-f1ed-4e65-a158-e31ff81d1907", "content-length": "26", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
13742025-10-03T21:15:40.320ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13752025-10-03T21:15:40.322ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
13762025-10-03T21:15:40.324ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13772025-10-03T21:15:40.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13782025-10-03T21:15:40.327ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13792025-10-03T21:15:40.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13802025-10-03T21:15:40.331ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13812025-10-03T21:15:40.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
13822025-10-03T21:15:40.334ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13832025-10-03T21:15:40.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13842025-10-03T21:15:40.336ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13852025-10-03T21:15:40.336ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
13862025-10-03T21:15:40.336ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 146.715636ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
13872025-10-03T21:15:40.337ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 147.825359ms iteration = 18 status = 0
13882025-10-03T21:15:40.338ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
13892025-10-03T21:15:40.338ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13902025-10-03T21:15:40.339ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f15b6b04-37a1-4eba-a8a7-6b3a4b7f01ef", "content-length": "26", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
13912025-10-03T21:15:40.339ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13922025-10-03T21:15:40.341ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13932025-10-03T21:15:40.343ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
13942025-10-03T21:15:40.343ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 153.158579ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
13952025-10-03T21:15:40.362ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13962025-10-03T21:15:40.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13972025-10-03T21:15:40.373ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13982025-10-03T21:15:40.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13992025-10-03T21:15:40.375ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14002025-10-03T21:15:40.378ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14012025-10-03T21:15:40.378ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14022025-10-03T21:15:40.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
14032025-10-03T21:15:40.380ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14042025-10-03T21:15:40.381ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14052025-10-03T21:15:40.381ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14062025-10-03T21:15:40.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
14072025-10-03T21:15:40.402ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
14082025-10-03T21:15:40.402ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14092025-10-03T21:15:40.403ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6dbac17-de75-4b69-8da3-5dfc8c41e4cb", "content-length": "372", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
14102025-10-03T21:15:40.403ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14112025-10-03T21:15:40.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14122025-10-03T21:15:40.413ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
14132025-10-03T21:15:40.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
14142025-10-03T21:15:40.415ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 227.185875ms iteration = 18 status = null
14152025-10-03T21:15:40.417ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
14162025-10-03T21:15:40.417ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
14172025-10-03T21:15:40.417ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "41422142-840e-4f8f-8ebd-5014e25f3013", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
14182025-10-03T21:15:40.417ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14192025-10-03T21:15:40.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14202025-10-03T21:15:40.420ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14212025-10-03T21:15:40.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14222025-10-03T21:15:40.423ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14232025-10-03T21:15:40.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14242025-10-03T21:15:40.427ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
14252025-10-03T21:15:40.427ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14262025-10-03T21:15:40.427ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7fafe9c5-0baf-4c92-86f6-db530d7839c7", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
14272025-10-03T21:15:40.429ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 239.882037ms iteration = 18 status = {}
14282025-10-03T21:15:40.433ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 244.672262ms iteration = 18 status = {}
14292025-10-03T21:15:40.458ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
14302025-10-03T21:15:40.461ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "482cdfe5-925f-45d3-b897-8e2f8ee74c7a", "content-length": "691", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
14312025-10-03T21:15:40.461ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
14322025-10-03T21:15:40.461ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
14332025-10-03T21:15:40.461ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
14342025-10-03T21:15:40.463ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 273.112853ms iteration = 18 status = {}
14352025-10-03T21:15:40.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14362025-10-03T21:15:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14372025-10-03T21:15:40.513ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
14382025-10-03T21:15:40.513ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14392025-10-03T21:15:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14402025-10-03T21:15:40.524ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
14412025-10-03T21:15:40.524ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
14422025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
14432025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
14442025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14452025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14462025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14472025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14482025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14492025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14502025-10-03T21:15:40.525ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
14512025-10-03T21:15:40.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14522025-10-03T21:15:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14532025-10-03T21:15:40.528ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
14542025-10-03T21:15:40.528ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14552025-10-03T21:15:40.673ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "981b737f-1333-4a8e-817f-87a5b0374a68", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
14562025-10-03T21:15:40.673ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
14572025-10-03T21:15:40.673ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14582025-10-03T21:15:40.674ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a61a4b2a-125d-4951-a038-b8a6b9beccee", "content-length": "682", "date": "Fri, 03 Oct 2025 21:15:40 GMT"} })
14592025-10-03T21:15:40.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14602025-10-03T21:15:40.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14612025-10-03T21:15:40.678ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14622025-10-03T21:15:40.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14632025-10-03T21:15:40.681ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14642025-10-03T21:15:40.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14652025-10-03T21:15:40.684ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14662025-10-03T21:15:40.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14672025-10-03T21:15:40.692ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 502.804926ms iteration = 18 status = {}
14682025-10-03T21:15:41.029ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 3
14692025-10-03T21:15:41.029ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
14702025-10-03T21:15:41.060ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
14712025-10-03T21:15:41.060ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:46941
14722025-10-03T21:15:41.060ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46941 req_id = 526846f6-6d43-43a7-ab79-a2d98b0fdae6 roles = RoleSet { roles: {} } uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
14732025-10-03T21:15:41.061ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46941 req_id = 526846f6-6d43-43a7-ab79-a2d98b0fdae6 resource = Database result = Ok(()) uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
14742025-10-03T21:15:41.098ZINFOnexus: assigned collector to new producer collector_id = d03f63a5-05d6-4fcc-b79e-7d63884498be file = nexus/src/app/oximeter.rs:76 producer_id = a5fae325-6f04-4795-bc31-fbf69228d92b
14752025-10-03T21:15:41.098ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 37925 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46941 req_id = 526846f6-6d43-43a7-ab79-a2d98b0fdae6 response_code = 201 uri = /metrics/producers
14762025-10-03T21:15:41.098ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "526846f6-6d43-43a7-ab79-a2d98b0fdae6", "content-length": "41", "date": "Fri, 03 Oct 2025 21:15:41 GMT"} })
14772025-10-03T21:15:41.100ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
14782025-10-03T21:15:41.223ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 378 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = b4f32217-8ff4-4854-afad-c6ee5bf82bc3 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
14792025-10-03T21:15:45.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 104 reason = Timeout
14802025-10-03T21:15:45.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
14812025-10-03T21:15:45.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
14822025-10-03T21:15:45.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14832025-10-03T21:15:45.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
14842025-10-03T21:15:45.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
14852025-10-03T21:15:45.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
14862025-10-03T21:15:45.197ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
14872025-10-03T21:15:45.197ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.597846ms iteration = 104 status = {"config_updated":false}
14882025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
14892025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
14902025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14912025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14922025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 105 reason = Timeout
14932025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
14942025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14952025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14962025-10-03T21:15:50.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
14972025-10-03T21:15:50.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14982025-10-03T21:15:50.192ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14992025-10-03T21:15:50.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15002025-10-03T21:15:50.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15012025-10-03T21:15:50.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15022025-10-03T21:15:50.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15032025-10-03T21:15:50.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15042025-10-03T21:15:50.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
15052025-10-03T21:15:50.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
15062025-10-03T21:15:50.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
15072025-10-03T21:15:50.197ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
15082025-10-03T21:15:50.197ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.319732ms iteration = 105 status = {"config_updated":false}
15092025-10-03T21:15:50.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15102025-10-03T21:15:50.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15112025-10-03T21:15:50.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15122025-10-03T21:15:50.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
15132025-10-03T21:15:50.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15142025-10-03T21:15:50.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15152025-10-03T21:15:50.218ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
15162025-10-03T21:15:50.218ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.38962ms iteration = 53 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
15172025-10-03T21:15:50.226ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.614014ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15182025-10-03T21:15:51.222ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 398 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 86f68228-d5b6-455c-8946-120375ac39d7 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
15192025-10-03T21:15:55.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 106 reason = Timeout
15202025-10-03T21:15:55.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
15212025-10-03T21:15:55.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
15222025-10-03T21:15:55.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15232025-10-03T21:15:55.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
15242025-10-03T21:15:55.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
15252025-10-03T21:15:55.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
15262025-10-03T21:15:55.196ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
15272025-10-03T21:15:55.196ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.430693ms iteration = 106 status = {"config_updated":false}
15282025-10-03T21:16:00.189ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
15292025-10-03T21:16:00.189ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
15302025-10-03T21:16:00.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15312025-10-03T21:16:00.189ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 107 reason = Timeout
15322025-10-03T21:16:00.189ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15332025-10-03T21:16:00.189ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
15342025-10-03T21:16:00.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15352025-10-03T21:16:00.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
15362025-10-03T21:16:00.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15372025-10-03T21:16:00.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15382025-10-03T21:16:00.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15392025-10-03T21:16:00.192ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15402025-10-03T21:16:00.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15412025-10-03T21:16:00.193ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.923984ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15422025-10-03T21:16:00.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
15432025-10-03T21:16:00.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
15442025-10-03T21:16:00.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15452025-10-03T21:16:00.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15462025-10-03T21:16:00.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
15472025-10-03T21:16:00.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15482025-10-03T21:16:00.196ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
15492025-10-03T21:16:00.196ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.17276ms iteration = 107 status = {"config_updated":false}
15502025-10-03T21:16:00.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15512025-10-03T21:16:00.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15522025-10-03T21:16:00.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15532025-10-03T21:16:00.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
15542025-10-03T21:16:00.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15552025-10-03T21:16:00.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15562025-10-03T21:16:00.212ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
15572025-10-03T21:16:00.212ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.830958ms iteration = 54 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
15582025-10-03T21:16:01.223ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 408 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 0e232acd-f8a8-48be-b971-2ddf933367f4 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
15592025-10-03T21:16:05.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 108 reason = Timeout
15602025-10-03T21:16:05.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
15612025-10-03T21:16:05.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
15622025-10-03T21:16:05.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15632025-10-03T21:16:05.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
15642025-10-03T21:16:05.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
15652025-10-03T21:16:05.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
15662025-10-03T21:16:05.197ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
15672025-10-03T21:16:05.197ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.244191ms iteration = 108 status = {"config_updated":false}
15682025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
15692025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
15702025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
15712025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
15722025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
15732025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15742025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
15752025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
15762025-10-03T21:16:10.189ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-10-03 21:06:10.189666747 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
15772025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
15782025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
15792025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
15802025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
15812025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
15822025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15832025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15842025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
15852025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15862025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
15872025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
15882025-10-03T21:16:10.189ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
15892025-10-03T21:16:10.189ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = ExternalNetworking step id = Ensure
15902025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15912025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 19 reason = Timeout
15922025-10-03T21:16:10.189ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
15932025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
15942025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
15952025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
15962025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
15972025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15982025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15992025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16002025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16012025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
16022025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16032025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
16042025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
16052025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
16062025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
16072025-10-03T21:16:10.190ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
16082025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16092025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
16102025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16112025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
16122025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
16132025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
16142025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
16152025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
16162025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
16172025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
16182025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
16192025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
16202025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
16212025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
16222025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
16232025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
16242025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16252025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
16262025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16272025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 10 reason = Timeout
16282025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
16292025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
16302025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
16312025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 10 reason = Timeout
16322025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
16332025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 109 reason = Timeout
16342025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
16352025-10-03T21:16:10.190ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
16362025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
16372025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
16382025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
16392025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16402025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 22 reason = Timeout
16412025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16422025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
16432025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16442025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
16452025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16462025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16472025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
16482025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16492025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
16502025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
16512025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16522025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16532025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
16542025-10-03T21:16:10.190ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16552025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 19 reason = Timeout
16562025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.92\u{b5}s iteration = 19 status = {"disabled":true,"errors":[],"sps":[]}
16572025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
16582025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16592025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
16602025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16612025-10-03T21:16:10.191ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2efcfb05-0c3d-45aa-b120-c334f3706be4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a7c1656-d513-4cc3-b2b0-8e31ad2b5545.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 5015, tv_nsec: 680613593 } })
16622025-10-03T21:16:10.191ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("486849a9-1733-4aa9-9b66-43ace2da0eea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 5015, tv_nsec: 680613603 } })
16632025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
16642025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16652025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
16662025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
16672025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
16682025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
16692025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
16702025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
16712025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16722025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
16732025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
16742025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16752025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
16762025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
16772025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
16782025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16792025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16802025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16812025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
16822025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16832025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16842025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
16852025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.095094ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
16862025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
16872025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.097744ms iteration = 10 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
16882025-10-03T21:16:10.191ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16892025-10-03T21:16:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16902025-10-03T21:16:10.192ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16912025-10-03T21:16:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16922025-10-03T21:16:10.192ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16932025-10-03T21:16:10.192ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16942025-10-03T21:16:10.192ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16952025-10-03T21:16:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16962025-10-03T21:16:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16972025-10-03T21:16:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16982025-10-03T21:16:10.194ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16992025-10-03T21:16:10.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17002025-10-03T21:16:10.194ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17012025-10-03T21:16:10.194ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17022025-10-03T21:16:10.194ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5015, tv_nsec: 683802819 } })
17032025-10-03T21:16:10.194ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17042025-10-03T21:16:10.195ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
17052025-10-03T21:16:10.195ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17062025-10-03T21:16:10.195ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-10-03 21:06:10.189666747 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
17072025-10-03T21:16:10.195ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.401281ms iteration = 10 status = {"expiration":"2025-10-03T21:06:10.189666747Z","pruned":[]}
17082025-10-03T21:16:10.195ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
17092025-10-03T21:16:10.195ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.496481ms iteration = 10 status = {"by_rx":{},"error":null}
17102025-10-03T21:16:10.196ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 5.680114ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
17112025-10-03T21:16:10.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
17122025-10-03T21:16:10.197ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17132025-10-03T21:16:10.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
17142025-10-03T21:16:10.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
17152025-10-03T21:16:10.198ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
17162025-10-03T21:16:10.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17172025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17182025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
17192025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17202025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17212025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17222025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17232025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
17242025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
17252025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17262025-10-03T21:16:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17272025-10-03T21:16:10.200ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.379557ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
17282025-10-03T21:16:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17292025-10-03T21:16:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17302025-10-03T21:16:10.200ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17312025-10-03T21:16:10.200ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 10.021783ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
17322025-10-03T21:16:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17332025-10-03T21:16:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
17342025-10-03T21:16:10.201ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17352025-10-03T21:16:10.202ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5015, tv_nsec: 691637358 } })
17362025-10-03T21:16:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17372025-10-03T21:16:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17382025-10-03T21:16:10.202ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17392025-10-03T21:16:10.202ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
17402025-10-03T21:16:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17412025-10-03T21:16:10.202ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17422025-10-03T21:16:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
17432025-10-03T21:16:10.203ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
17442025-10-03T21:16:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
17452025-10-03T21:16:10.204ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17462025-10-03T21:16:10.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17472025-10-03T21:16:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
17482025-10-03T21:16:10.204ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
17492025-10-03T21:16:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
17502025-10-03T21:16:10.205ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17512025-10-03T21:16:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
17522025-10-03T21:16:10.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
17532025-10-03T21:16:10.207ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17542025-10-03T21:16:10.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17552025-10-03T21:16:10.207ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17562025-10-03T21:16:10.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17572025-10-03T21:16:10.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17582025-10-03T21:16:10.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
17592025-10-03T21:16:10.208ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
17602025-10-03T21:16:10.208ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
17612025-10-03T21:16:10.208ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 17.994973ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
17622025-10-03T21:16:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17632025-10-03T21:16:10.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17642025-10-03T21:16:10.209ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17652025-10-03T21:16:10.210ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17662025-10-03T21:16:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17672025-10-03T21:16:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
17682025-10-03T21:16:10.210ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17692025-10-03T21:16:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17702025-10-03T21:16:10.212ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17712025-10-03T21:16:10.212ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 22.204081ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
17722025-10-03T21:16:10.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
17732025-10-03T21:16:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
17742025-10-03T21:16:10.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17752025-10-03T21:16:10.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17762025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
17772025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
17782025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17792025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
17802025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17812025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17822025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
17832025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
17842025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
17852025-10-03T21:16:10.217ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17862025-10-03T21:16:10.217ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5015, tv_nsec: 707289434 } })
17872025-10-03T21:16:10.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17882025-10-03T21:16:10.218ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
17892025-10-03T21:16:10.219ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17902025-10-03T21:16:10.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17912025-10-03T21:16:10.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17922025-10-03T21:16:10.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17932025-10-03T21:16:10.220ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 29.189109ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
17942025-10-03T21:16:10.220ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17952025-10-03T21:16:10.220ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
17962025-10-03T21:16:10.220ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 29.619304ms iteration = 109 status = {"config_updated":false}
17972025-10-03T21:16:10.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17982025-10-03T21:16:10.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
17992025-10-03T21:16:10.223ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18002025-10-03T21:16:10.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18012025-10-03T21:16:10.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18022025-10-03T21:16:10.225ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18032025-10-03T21:16:10.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18042025-10-03T21:16:10.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18052025-10-03T21:16:10.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18062025-10-03T21:16:10.228ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18072025-10-03T21:16:10.228ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18082025-10-03T21:16:10.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
18092025-10-03T21:16:10.232ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18102025-10-03T21:16:10.232ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
18112025-10-03T21:16:10.232ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18122025-10-03T21:16:10.234ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 42.939074ms iteration = 10 status = {"dispatched":[],"errors":[],"glob_version":"195.0.0","globs_reprocessed":{},"no_receivers":[]}
18132025-10-03T21:16:10.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18142025-10-03T21:16:10.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18152025-10-03T21:16:10.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18162025-10-03T21:16:10.239ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18172025-10-03T21:16:10.239ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18182025-10-03T21:16:10.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18192025-10-03T21:16:10.246ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18202025-10-03T21:16:10.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18212025-10-03T21:16:10.250ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
18222025-10-03T21:16:10.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
18232025-10-03T21:16:10.253ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18242025-10-03T21:16:10.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
18252025-10-03T21:16:10.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
18262025-10-03T21:16:10.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18272025-10-03T21:16:10.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18282025-10-03T21:16:10.257ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 67.033817ms iteration = 5 status = {"nsuccess":5}
18292025-10-03T21:16:10.259ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
18302025-10-03T21:16:10.263ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18312025-10-03T21:16:10.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
18322025-10-03T21:16:10.265ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 74.750964ms iteration = 29 status = {"physical_disks_added":0}
18332025-10-03T21:16:10.267ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
18342025-10-03T21:16:10.268ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18352025-10-03T21:16:10.271ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
18362025-10-03T21:16:10.271ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18372025-10-03T21:16:10.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
18382025-10-03T21:16:10.273ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
18392025-10-03T21:16:10.279ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
18402025-10-03T21:16:10.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 92.576625ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
18412025-10-03T21:16:10.287ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
18422025-10-03T21:16:10.291ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18432025-10-03T21:16:10.291ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
18442025-10-03T21:16:10.291ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18452025-10-03T21:16:10.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18462025-10-03T21:16:10.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
18472025-10-03T21:16:10.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18482025-10-03T21:16:10.295ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
18492025-10-03T21:16:10.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18502025-10-03T21:16:10.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18512025-10-03T21:16:10.298ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
18522025-10-03T21:16:10.298ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 107.726116ms iteration = 55 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
18532025-10-03T21:16:10.303ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18542025-10-03T21:16:10.303ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
18552025-10-03T21:16:10.303ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18562025-10-03T21:16:10.304ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
18572025-10-03T21:16:10.304ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
18582025-10-03T21:16:10.304ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 113.923566ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
18592025-10-03T21:16:10.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
18602025-10-03T21:16:10.305ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
18612025-10-03T21:16:10.305ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
18622025-10-03T21:16:10.305ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 115.102419ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
18632025-10-03T21:16:10.307ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
18642025-10-03T21:16:10.307ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18652025-10-03T21:16:10.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
18662025-10-03T21:16:10.309ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
18672025-10-03T21:16:10.311ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
18682025-10-03T21:16:10.311ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18692025-10-03T21:16:10.312ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = SledList step id = Fetch
18702025-10-03T21:16:10.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
18712025-10-03T21:16:10.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18722025-10-03T21:16:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
18732025-10-03T21:16:10.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18742025-10-03T21:16:10.315ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 124.703798ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18752025-10-03T21:16:10.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18762025-10-03T21:16:10.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18772025-10-03T21:16:10.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18782025-10-03T21:16:10.319ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
18792025-10-03T21:16:10.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
18802025-10-03T21:16:10.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
18812025-10-03T21:16:10.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18822025-10-03T21:16:10.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
18832025-10-03T21:16:10.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18842025-10-03T21:16:10.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18852025-10-03T21:16:10.324ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18862025-10-03T21:16:10.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
18872025-10-03T21:16:10.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
18882025-10-03T21:16:10.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18892025-10-03T21:16:10.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18902025-10-03T21:16:10.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
18912025-10-03T21:16:10.326ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18922025-10-03T21:16:10.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
18932025-10-03T21:16:10.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18942025-10-03T21:16:10.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18952025-10-03T21:16:10.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
18962025-10-03T21:16:10.330ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18972025-10-03T21:16:10.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18982025-10-03T21:16:10.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18992025-10-03T21:16:10.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19002025-10-03T21:16:10.335ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
19012025-10-03T21:16:10.335ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 145.412442ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
19022025-10-03T21:16:10.340ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19032025-10-03T21:16:10.340ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19042025-10-03T21:16:10.340ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19052025-10-03T21:16:10.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19062025-10-03T21:16:10.340ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19072025-10-03T21:16:10.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19082025-10-03T21:16:10.343ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
19092025-10-03T21:16:10.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19102025-10-03T21:16:10.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19112025-10-03T21:16:10.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19122025-10-03T21:16:10.352ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19132025-10-03T21:16:10.352ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
19142025-10-03T21:16:10.353ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
19152025-10-03T21:16:10.353ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19162025-10-03T21:16:10.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19172025-10-03T21:16:10.355ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
19182025-10-03T21:16:10.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19192025-10-03T21:16:10.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19202025-10-03T21:16:10.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19212025-10-03T21:16:10.357ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19222025-10-03T21:16:10.357ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19232025-10-03T21:16:10.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19242025-10-03T21:16:10.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19252025-10-03T21:16:10.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19262025-10-03T21:16:10.363ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19272025-10-03T21:16:10.363ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19282025-10-03T21:16:10.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19292025-10-03T21:16:10.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19302025-10-03T21:16:10.365ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
19312025-10-03T21:16:10.365ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
19322025-10-03T21:16:10.365ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-10-03 21:07:14.470425 UTC dns_group = external generation = 2
19332025-10-03T21:16:10.365ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 175.999507ms iteration = 10 status = {"generation":2}
19342025-10-03T21:16:10.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19352025-10-03T21:16:10.366ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-10-03T21:07:04.185535Z, time_modified: 2025-10-03T21:07:04.185535Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }) }
19362025-10-03T21:16:10.366ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 177.15302ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]}},"default_endpoint":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
19372025-10-03T21:16:10.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19382025-10-03T21:16:10.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
19392025-10-03T21:16:10.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19402025-10-03T21:16:10.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19412025-10-03T21:16:10.374ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
19422025-10-03T21:16:10.376ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 185.476644ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19432025-10-03T21:16:10.377ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19442025-10-03T21:16:10.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19452025-10-03T21:16:10.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19462025-10-03T21:16:10.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19472025-10-03T21:16:10.381ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 190.565112ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
19482025-10-03T21:16:10.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19492025-10-03T21:16:10.382ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19502025-10-03T21:16:10.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19512025-10-03T21:16:10.385ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19522025-10-03T21:16:10.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
19532025-10-03T21:16:10.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19542025-10-03T21:16:10.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19552025-10-03T21:16:10.388ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19562025-10-03T21:16:10.388ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
19572025-10-03T21:16:10.388ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 197.422929ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
19582025-10-03T21:16:10.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19592025-10-03T21:16:10.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
19602025-10-03T21:16:10.391ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19612025-10-03T21:16:10.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19622025-10-03T21:16:10.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
19632025-10-03T21:16:10.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19642025-10-03T21:16:10.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19652025-10-03T21:16:10.396ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
19662025-10-03T21:16:10.396ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19672025-10-03T21:16:10.396ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 205.57545ms iteration = 19 status = 0
19682025-10-03T21:16:10.396ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "07068bc2-100b-4d41-8905-b4d0c043000b", "content-length": "26", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
19692025-10-03T21:16:10.396ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19702025-10-03T21:16:10.398ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19712025-10-03T21:16:10.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19722025-10-03T21:16:10.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19732025-10-03T21:16:10.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19742025-10-03T21:16:10.407ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
19752025-10-03T21:16:10.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19762025-10-03T21:16:10.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19772025-10-03T21:16:10.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19782025-10-03T21:16:10.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19792025-10-03T21:16:10.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19802025-10-03T21:16:10.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19812025-10-03T21:16:10.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19822025-10-03T21:16:10.422ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
19832025-10-03T21:16:10.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19842025-10-03T21:16:10.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19852025-10-03T21:16:10.425ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
19862025-10-03T21:16:10.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19872025-10-03T21:16:10.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
19882025-10-03T21:16:10.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19892025-10-03T21:16:10.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19902025-10-03T21:16:10.432ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
19912025-10-03T21:16:10.432ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19922025-10-03T21:16:10.432ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b00483fc-b9eb-4512-ba2a-5894d5d372f8", "content-length": "26", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
19932025-10-03T21:16:10.432ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19942025-10-03T21:16:10.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19952025-10-03T21:16:10.434ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
19962025-10-03T21:16:10.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19972025-10-03T21:16:10.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19982025-10-03T21:16:10.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19992025-10-03T21:16:10.441ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
20002025-10-03T21:16:10.441ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
20012025-10-03T21:16:10.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20022025-10-03T21:16:10.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20032025-10-03T21:16:10.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20042025-10-03T21:16:10.444ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20052025-10-03T21:16:10.448ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
20062025-10-03T21:16:10.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20072025-10-03T21:16:10.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20082025-10-03T21:16:10.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20092025-10-03T21:16:10.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20102025-10-03T21:16:10.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20112025-10-03T21:16:10.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20122025-10-03T21:16:10.451ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
20132025-10-03T21:16:10.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
20142025-10-03T21:16:10.452ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
20152025-10-03T21:16:10.452ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
20162025-10-03T21:16:10.452ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-10-03 21:07:12.445842 UTC dns_group = internal generation = 1
20172025-10-03T21:16:10.452ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 263.034399ms iteration = 10 status = {"generation":1}
20182025-10-03T21:16:10.454ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
20192025-10-03T21:16:10.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20202025-10-03T21:16:10.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20212025-10-03T21:16:10.455ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
20222025-10-03T21:16:10.455ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 265.318795ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
20232025-10-03T21:16:10.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20242025-10-03T21:16:10.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20252025-10-03T21:16:10.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20262025-10-03T21:16:10.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20272025-10-03T21:16:10.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20282025-10-03T21:16:10.461ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20292025-10-03T21:16:10.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
20302025-10-03T21:16:10.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
20312025-10-03T21:16:10.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20322025-10-03T21:16:10.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20332025-10-03T21:16:10.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20342025-10-03T21:16:10.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20352025-10-03T21:16:10.467ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = DeployNexusRecords step id = Ensure
20362025-10-03T21:16:10.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20372025-10-03T21:16:10.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20382025-10-03T21:16:10.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20392025-10-03T21:16:10.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20402025-10-03T21:16:10.471ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
20412025-10-03T21:16:10.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20422025-10-03T21:16:10.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20432025-10-03T21:16:10.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20442025-10-03T21:16:10.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20452025-10-03T21:16:10.475ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
20462025-10-03T21:16:10.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20472025-10-03T21:16:10.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20482025-10-03T21:16:10.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20492025-10-03T21:16:10.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20502025-10-03T21:16:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20512025-10-03T21:16:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
20522025-10-03T21:16:10.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20532025-10-03T21:16:10.483ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = SledAgent step id = Ensure
20542025-10-03T21:16:10.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
20552025-10-03T21:16:10.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20562025-10-03T21:16:10.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20572025-10-03T21:16:10.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20582025-10-03T21:16:10.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20592025-10-03T21:16:10.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20602025-10-03T21:16:10.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
20612025-10-03T21:16:10.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20622025-10-03T21:16:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20632025-10-03T21:16:10.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20642025-10-03T21:16:10.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20652025-10-03T21:16:10.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20662025-10-03T21:16:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
20672025-10-03T21:16:10.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20682025-10-03T21:16:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20692025-10-03T21:16:10.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20702025-10-03T21:16:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20712025-10-03T21:16:10.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20722025-10-03T21:16:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
20732025-10-03T21:16:10.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20742025-10-03T21:16:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20752025-10-03T21:16:10.512ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
20762025-10-03T21:16:10.512ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
20772025-10-03T21:16:10.513ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 5a4a3a10-8efd-40e6-9067-9c89e5fc0979 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
20782025-10-03T21:16:10.513ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
20792025-10-03T21:16:10.513ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 322.427339ms iteration = 22 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
20802025-10-03T21:16:10.529ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20812025-10-03T21:16:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
20822025-10-03T21:16:10.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20832025-10-03T21:16:10.534ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20842025-10-03T21:16:10.534ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20852025-10-03T21:16:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
20862025-10-03T21:16:10.536ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20872025-10-03T21:16:10.538ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20882025-10-03T21:16:10.538ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20892025-10-03T21:16:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
20902025-10-03T21:16:10.543ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
20912025-10-03T21:16:10.543ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20922025-10-03T21:16:10.543ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "43478194-326f-40e9-a677-549090728cb8", "content-length": "372", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
20932025-10-03T21:16:10.543ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20942025-10-03T21:16:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20952025-10-03T21:16:10.555ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
20962025-10-03T21:16:10.555ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20972025-10-03T21:16:10.555ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "be5b4819-3563-4fa3-8aea-b00db2e0697d", "content-length": "26", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
20982025-10-03T21:16:10.555ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20992025-10-03T21:16:10.563ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21002025-10-03T21:16:10.564ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
21012025-10-03T21:16:10.564ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
21022025-10-03T21:16:10.564ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e285bfba-a784-4121-87b6-640522631d13", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
21032025-10-03T21:16:10.564ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21042025-10-03T21:16:10.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21052025-10-03T21:16:10.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21062025-10-03T21:16:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21072025-10-03T21:16:10.573ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21082025-10-03T21:16:10.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21092025-10-03T21:16:10.577ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
21102025-10-03T21:16:10.577ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21112025-10-03T21:16:10.578ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3a9e5f69-5091-4ac9-a901-51625b187ca6", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
21122025-10-03T21:16:10.580ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 389.647078ms iteration = 19 status = {}
21132025-10-03T21:16:10.587ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
21142025-10-03T21:16:10.588ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "23c5a90e-cadf-4903-9755-9f614a56a5bf", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
21152025-10-03T21:16:10.588ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
21162025-10-03T21:16:10.590ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f5b0201b-610a-45b8-bb6d-de50df31d7dc", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
21172025-10-03T21:16:10.590ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
21182025-10-03T21:16:10.594ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
21192025-10-03T21:16:10.594ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 404.626927ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
21202025-10-03T21:16:10.635ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
21212025-10-03T21:16:10.671ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
21222025-10-03T21:16:10.672ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "eca69a50-4e8f-4fea-a39b-2c43fea5af34", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} }) sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
21232025-10-03T21:16:10.675ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = FirewallRules step id = Ensure
21242025-10-03T21:16:10.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21252025-10-03T21:16:10.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21262025-10-03T21:16:10.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21272025-10-03T21:16:10.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21282025-10-03T21:16:10.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21292025-10-03T21:16:10.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21302025-10-03T21:16:10.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21312025-10-03T21:16:10.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21322025-10-03T21:16:10.684ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
21332025-10-03T21:16:10.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
21342025-10-03T21:16:10.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21352025-10-03T21:16:10.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21362025-10-03T21:16:10.686ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 496.870708ms iteration = 19 status = null
21372025-10-03T21:16:10.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21382025-10-03T21:16:10.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21392025-10-03T21:16:10.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21402025-10-03T21:16:10.694ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
21412025-10-03T21:16:10.694ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21422025-10-03T21:16:10.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21432025-10-03T21:16:10.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21442025-10-03T21:16:10.694ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0863fc7-f8bb-4d09-969b-916c3fbc1920", "content-length": "691", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
21452025-10-03T21:16:10.694ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
21462025-10-03T21:16:10.694ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
21472025-10-03T21:16:10.694ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
21482025-10-03T21:16:10.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21492025-10-03T21:16:10.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21502025-10-03T21:16:10.697ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 506.722049ms iteration = 19 status = {}
21512025-10-03T21:16:10.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21522025-10-03T21:16:10.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21532025-10-03T21:16:10.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21542025-10-03T21:16:10.706ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21552025-10-03T21:16:10.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21562025-10-03T21:16:10.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21572025-10-03T21:16:10.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21582025-10-03T21:16:10.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21592025-10-03T21:16:10.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21602025-10-03T21:16:10.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21612025-10-03T21:16:10.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21622025-10-03T21:16:10.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21632025-10-03T21:16:10.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21642025-10-03T21:16:10.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21652025-10-03T21:16:10.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21662025-10-03T21:16:10.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21672025-10-03T21:16:10.718ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 528.986311ms iteration = 19 status = {}
21682025-10-03T21:16:10.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21692025-10-03T21:16:10.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21702025-10-03T21:16:10.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21712025-10-03T21:16:10.724ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21722025-10-03T21:16:10.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21732025-10-03T21:16:10.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21742025-10-03T21:16:10.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21752025-10-03T21:16:10.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21762025-10-03T21:16:10.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21772025-10-03T21:16:10.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21782025-10-03T21:16:10.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21792025-10-03T21:16:10.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21802025-10-03T21:16:10.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21812025-10-03T21:16:10.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21822025-10-03T21:16:10.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21832025-10-03T21:16:10.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21842025-10-03T21:16:10.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21852025-10-03T21:16:10.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21862025-10-03T21:16:10.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21872025-10-03T21:16:10.740ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
21882025-10-03T21:16:10.741ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "dfed07b9-ae01-4d3d-8d44-e67096fdea04", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
21892025-10-03T21:16:10.741ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
21902025-10-03T21:16:10.743ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "53a36da1-204f-48e3-b0a9-2056e0eb4e1f", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
21912025-10-03T21:16:10.743ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
21922025-10-03T21:16:10.744ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
21932025-10-03T21:16:10.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21942025-10-03T21:16:10.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21952025-10-03T21:16:10.745ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "b41a2a0d-6764-4f9c-a03d-1e213c0ded06", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
21962025-10-03T21:16:10.745ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
21972025-10-03T21:16:10.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21982025-10-03T21:16:10.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21992025-10-03T21:16:10.747ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
22002025-10-03T21:16:10.747ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 558.16721ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
22012025-10-03T21:16:10.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22022025-10-03T21:16:10.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22032025-10-03T21:16:10.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22042025-10-03T21:16:10.754ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22052025-10-03T21:16:10.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22062025-10-03T21:16:10.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22072025-10-03T21:16:10.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22082025-10-03T21:16:10.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22092025-10-03T21:16:10.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22102025-10-03T21:16:10.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22112025-10-03T21:16:10.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22122025-10-03T21:16:10.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22132025-10-03T21:16:10.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22142025-10-03T21:16:10.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22152025-10-03T21:16:10.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22162025-10-03T21:16:10.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22172025-10-03T21:16:10.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22182025-10-03T21:16:10.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22192025-10-03T21:16:10.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22202025-10-03T21:16:10.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22212025-10-03T21:16:10.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22222025-10-03T21:16:10.778ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
22232025-10-03T21:16:10.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22242025-10-03T21:16:10.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22252025-10-03T21:16:10.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22262025-10-03T21:16:10.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22272025-10-03T21:16:10.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22282025-10-03T21:16:10.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22292025-10-03T21:16:10.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22302025-10-03T21:16:10.786ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
22312025-10-03T21:16:10.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22322025-10-03T21:16:10.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22332025-10-03T21:16:10.792ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
22342025-10-03T21:16:10.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22352025-10-03T21:16:10.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22362025-10-03T21:16:10.793ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22372025-10-03T21:16:10.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22382025-10-03T21:16:10.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22392025-10-03T21:16:10.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
22402025-10-03T21:16:10.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22412025-10-03T21:16:10.796ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
22422025-10-03T21:16:10.796ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22432025-10-03T21:16:10.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22442025-10-03T21:16:10.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22452025-10-03T21:16:10.797ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
22462025-10-03T21:16:10.797ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
22472025-10-03T21:16:10.797ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
22482025-10-03T21:16:10.805ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
22492025-10-03T21:16:10.805ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
22502025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
22512025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
22522025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22532025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22542025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22552025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
22562025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22572025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
22582025-10-03T21:16:10.806ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
22592025-10-03T21:16:10.806ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22602025-10-03T21:16:10.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22612025-10-03T21:16:10.808ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
22622025-10-03T21:16:10.808ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
22632025-10-03T21:16:10.859ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), time_created: 2025-10-03T21:07:05.994Z, time_modified: 2025-10-03T21:07:05.994Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
22642025-10-03T21:16:10.859ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
22652025-10-03T21:16:10.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22662025-10-03T21:16:10.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22672025-10-03T21:16:10.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22682025-10-03T21:16:10.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22692025-10-03T21:16:10.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22702025-10-03T21:16:10.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
22712025-10-03T21:16:10.898ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22722025-10-03T21:16:10.902ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "c1feca0d-52f7-41ef-a490-ce23e96c8273", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
22732025-10-03T21:16:10.903ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
22742025-10-03T21:16:10.903ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = Dns step id = Ensure
22752025-10-03T21:16:10.903ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
22762025-10-03T21:16:10.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22772025-10-03T21:16:10.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22782025-10-03T21:16:10.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22792025-10-03T21:16:10.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22802025-10-03T21:16:10.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22812025-10-03T21:16:10.909ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
22822025-10-03T21:16:10.910ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
22832025-10-03T21:16:10.910ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
22842025-10-03T21:16:10.910ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
22852025-10-03T21:16:10.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22862025-10-03T21:16:10.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22872025-10-03T21:16:10.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22882025-10-03T21:16:10.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
22892025-10-03T21:16:10.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22902025-10-03T21:16:10.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22912025-10-03T21:16:10.917ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
22922025-10-03T21:16:10.917ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22932025-10-03T21:16:10.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22942025-10-03T21:16:10.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22952025-10-03T21:16:10.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22962025-10-03T21:16:10.922ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
22972025-10-03T21:16:10.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22982025-10-03T21:16:10.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22992025-10-03T21:16:10.924ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
23002025-10-03T21:16:10.924ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
23012025-10-03T21:16:10.924ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
23022025-10-03T21:16:10.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23032025-10-03T21:16:10.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23042025-10-03T21:16:10.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23052025-10-03T21:16:10.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23062025-10-03T21:16:10.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23072025-10-03T21:16:10.930ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
23082025-10-03T21:16:10.931ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
23092025-10-03T21:16:10.931ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
23102025-10-03T21:16:10.931ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
23112025-10-03T21:16:10.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23122025-10-03T21:16:10.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23132025-10-03T21:16:10.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23142025-10-03T21:16:10.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
23152025-10-03T21:16:10.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23162025-10-03T21:16:10.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23172025-10-03T21:16:10.938ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
23182025-10-03T21:16:10.938ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
23192025-10-03T21:16:10.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23202025-10-03T21:16:10.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23212025-10-03T21:16:10.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23222025-10-03T21:16:10.942ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
23232025-10-03T21:16:10.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23242025-10-03T21:16:10.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23252025-10-03T21:16:10.944ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
23262025-10-03T21:16:10.944ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
23272025-10-03T21:16:10.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23282025-10-03T21:16:10.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23292025-10-03T21:16:10.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23302025-10-03T21:16:10.947ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
23312025-10-03T21:16:10.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23322025-10-03T21:16:10.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23332025-10-03T21:16:10.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23342025-10-03T21:16:10.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23352025-10-03T21:16:10.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23362025-10-03T21:16:10.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23372025-10-03T21:16:10.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23382025-10-03T21:16:10.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
23392025-10-03T21:16:10.956ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
23402025-10-03T21:16:10.956ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
23412025-10-03T21:16:10.956ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = OmicronZones step id = Cleanup
23422025-10-03T21:16:10.956ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = OmicronZones step id = Cleanup
23432025-10-03T21:16:10.956ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = PhysicalDisks step id = Cleanup
23442025-10-03T21:16:10.956ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = Clickhouse step id = Ensure
23452025-10-03T21:16:10.956ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = Clickhouse step id = Ensure
23462025-10-03T21:16:10.992ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
23472025-10-03T21:16:10.996ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "679de700-d5ed-4671-a270-a39dade9ba6a", "date": "Fri, 03 Oct 2025 21:16:10 GMT"} })
23482025-10-03T21:16:10.997ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = SupportBundles step id = Cleanup
23492025-10-03T21:16:10.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23502025-10-03T21:16:10.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23512025-10-03T21:16:10.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23522025-10-03T21:16:11.001ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
23532025-10-03T21:16:11.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23542025-10-03T21:16:11.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23552025-10-03T21:16:11.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23562025-10-03T21:16:11.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23572025-10-03T21:16:11.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23582025-10-03T21:16:11.006ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
23592025-10-03T21:16:11.009ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = OmicronZones step id = Cleanup
23602025-10-03T21:16:11.010ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
23612025-10-03T21:16:11.010ZDEBGnexus (BackgroundTasks): re-assign sagas: found Nexus instances background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
23622025-10-03T21:16:11.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23632025-10-03T21:16:11.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23642025-10-03T21:16:11.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23652025-10-03T21:16:11.013ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
23662025-10-03T21:16:11.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23672025-10-03T21:16:11.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23682025-10-03T21:16:11.047ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:54 nexus_zone_ids = []
23692025-10-03T21:16:11.047ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(675bea75-8e2d-4986-86f2-b590a30bb084 (blueprint), false)
23702025-10-03T21:16:11.047ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = Cockroach step id = Ensure
23712025-10-03T21:16:11.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23722025-10-03T21:16:11.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23732025-10-03T21:16:11.054ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "adfd09eb-0e9e-4e08-a7c6-dcec53bf0814", "date": "Fri, 03 Oct 2025 21:16:11 GMT"} })
23742025-10-03T21:16:11.054ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
23752025-10-03T21:16:11.054ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23762025-10-03T21:16:11.061ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "87c0ec7a-f6ce-4995-95a9-a7fe95fd9d6c", "content-length": "682", "date": "Fri, 03 Oct 2025 21:16:11 GMT"} })
23772025-10-03T21:16:11.061ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23782025-10-03T21:16:11.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23792025-10-03T21:16:11.064ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23802025-10-03T21:16:11.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23812025-10-03T21:16:11.068ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23822025-10-03T21:16:11.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23832025-10-03T21:16:11.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23842025-10-03T21:16:11.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23852025-10-03T21:16:11.078ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 888.001544ms iteration = 19 status = {}
23862025-10-03T21:16:11.106ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
23872025-10-03T21:16:11.106ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3a5a042b-e1bc-42be-80a3-7bab942681d4 step component = MgsUpdates step id = Ensure
23882025-10-03T21:16:11.106ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 29 reason = Dependency
23892025-10-03T21:16:11.106ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 11 reason = Dependency
23902025-10-03T21:16:11.106ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23912025-10-03T21:16:11.106ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
23922025-10-03T21:16:11.107ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 917.406435ms iteration = 11 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4840,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":122632544,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":122632494,"secs":0}},"event_index":1,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":122662005,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":155321554,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":155321514,"secs":0}},"event_index":2,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":278002598,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":15299342,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":15299302,"secs":0}},"event_index":3,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":293321171,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":192068407,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":192068367,"secs":0}},"event_index":4,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":485412619,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":228719111,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":228719071,"secs":0}},"event_index":5,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":714148221,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":52610494,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":52610445,"secs":0}},"event_index":6,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":766768495,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4401,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4351,"secs":0}},"event_index":7,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":766784276,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2790,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2750,"secs":0}},"event_index":8,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":766789566,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2920,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2880,"secs":0}},"event_index":9,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":766794486,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2120,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2080,"secs":0}},"event_index":10,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":766798526,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41153764,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":41153724,"secs":0}},"event_index":11,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":807953330,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":12229327,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":12229287,"secs":0}},"event_index":12,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":820194669,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":37037128,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":37037078,"secs":0}},"event_index":13,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":857242467,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":59778565,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":59778235,"secs":0}},"event_index":14,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":917032752,"secs":0}},{"data":{"attempt_elapsed":{"nanos":5670,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5630,"secs":0}},"event_index":15,"execution_id":"c4096408-db89-4cdd-a2d0-b49d0331f5a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":917047912,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"675bea75-8e2d-4986-86f2-b590a30bb084"}
23932025-10-03T21:16:11.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
23942025-10-03T21:16:11.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23952025-10-03T21:16:11.108ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23962025-10-03T21:16:11.109ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23972025-10-03T21:16:11.111ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
23982025-10-03T21:16:11.111ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23992025-10-03T21:16:11.111ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
24002025-10-03T21:16:11.111ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24012025-10-03T21:16:11.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24022025-10-03T21:16:11.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24032025-10-03T21:16:11.113ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
24042025-10-03T21:16:11.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24052025-10-03T21:16:11.115ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(f045050f-9d07-4886-92bb-95ce48733e72, 0), (ff7faa4e-ee9d-427a-924f-4d8aad298aef, 0), (d9bf6af8-0c02-4ab9-80a0-fdc64bfdcdbd, 0), (dec5ed19-893d-4e35-ab67-0e9cbeb66973, 0)] collection_id = f045050f-9d07-4886-92bb-95ce48733e72
24062025-10-03T21:16:11.115ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24072025-10-03T21:16:11.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24082025-10-03T21:16:11.115ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24092025-10-03T21:16:11.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
24102025-10-03T21:16:11.117ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
24112025-10-03T21:16:11.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24122025-10-03T21:16:11.119ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
24132025-10-03T21:16:11.119ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.359149ms iteration = 29 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
24142025-10-03T21:16:11.223ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 501 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = c8582658-83a2-46e3-badd-1bb97a903cf9 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
24152025-10-03T21:16:11.832ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = f045050f-9d07-4886-92bb-95ce48733e72 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
24162025-10-03T21:16:11.832ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24172025-10-03T21:16:11.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24182025-10-03T21:16:11.844ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(ff7faa4e-ee9d-427a-924f-4d8aad298aef, 0), (d9bf6af8-0c02-4ab9-80a0-fdc64bfdcdbd, 0), (dec5ed19-893d-4e35-ab67-0e9cbeb66973, 0), (8a0d3c05-6d9a-45d0-95fb-f2b68709aa25, 0)] collection_id = ff7faa4e-ee9d-427a-924f-4d8aad298aef
24192025-10-03T21:16:11.844ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24202025-10-03T21:16:11.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24212025-10-03T21:16:12.533ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = ff7faa4e-ee9d-427a-924f-4d8aad298aef file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
24222025-10-03T21:16:12.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24232025-10-03T21:16:12.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24242025-10-03T21:16:12.582ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(d9bf6af8-0c02-4ab9-80a0-fdc64bfdcdbd, 0), (dec5ed19-893d-4e35-ab67-0e9cbeb66973, 0), (8a0d3c05-6d9a-45d0-95fb-f2b68709aa25, 0), (e33257de-ae23-49ef-9065-e527da3bdcfa, 0)] collection_id = d9bf6af8-0c02-4ab9-80a0-fdc64bfdcdbd
24252025-10-03T21:16:12.582ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24262025-10-03T21:16:12.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24272025-10-03T21:16:13.544ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = d9bf6af8-0c02-4ab9-80a0-fdc64bfdcdbd file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
24282025-10-03T21:16:13.544ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24292025-10-03T21:16:13.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24302025-10-03T21:16:13.555ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(dec5ed19-893d-4e35-ab67-0e9cbeb66973, 0), (8a0d3c05-6d9a-45d0-95fb-f2b68709aa25, 0), (e33257de-ae23-49ef-9065-e527da3bdcfa, 0)]
24312025-10-03T21:16:13.556ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5019, tv_nsec: 46227684 } })
24322025-10-03T21:16:13.588ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("26cdd4d2-d0ac-43df-bf3e-350fd160efec.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41cf4311-4781-41d3-b8bc-181c344891b0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b55c423c-6cde-4185-879d-3a1c4cf01de4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b807e991-1a36-4877-8937-3efb3994dbc2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::3)) }], valid_until: Instant { tv_sec: 5019, tv_nsec: 78194464 } })
24332025-10-03T21:16:13.589ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
24342025-10-03T21:16:13.589ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
24352025-10-03T21:16:13.589ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
24362025-10-03T21:16:13.589ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0b74370-1c6a-416f-88fc-6eb973fe79b2", "content-length": "476", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24372025-10-03T21:16:13.589ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
24382025-10-03T21:16:13.590ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa00cc77-cc35-414a-a3d3-3d3b9fbad27a", "content-length": "734", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24392025-10-03T21:16:13.590ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
24402025-10-03T21:16:13.590ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0f1e17c-6b69-4b2d-b61b-8ba86a48d5a1", "content-length": "111", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24412025-10-03T21:16:13.590ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
24422025-10-03T21:16:13.591ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "262e1c7c-e40e-4fec-991d-2d3a44af2f82", "content-length": "111", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24432025-10-03T21:16:13.591ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
24442025-10-03T21:16:13.591ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b76ce0b8-87ce-4016-8b97-5e3398376c36", "content-length": "163", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24452025-10-03T21:16:13.591ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
24462025-10-03T21:16:13.591ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b74b27ef-437d-43fc-a480-0c4ee8d1f485", "content-length": "163", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24472025-10-03T21:16:13.591ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
24482025-10-03T21:16:13.592ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8707544e-6272-4a84-b66b-9029d6fff25e", "content-length": "166", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24492025-10-03T21:16:13.592ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
24502025-10-03T21:16:13.592ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7463e746-1f06-48e7-9e97-2141cf434c57", "content-length": "166", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24512025-10-03T21:16:13.592ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
24522025-10-03T21:16:13.592ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b81885ac-81e2-4eee-8445-a1a0e8e5a0fe", "content-length": "702", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24532025-10-03T21:16:13.592ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
24542025-10-03T21:16:13.592ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b90cd213-90f7-4ee5-a507-6124c8232f90", "content-length": "727", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24552025-10-03T21:16:13.592ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
24562025-10-03T21:16:13.593ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7d06830-84ff-4628-98a3-f40544a6666a", "content-length": "729", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24572025-10-03T21:16:13.593ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
24582025-10-03T21:16:13.593ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37b3c8a7-b630-4d70-81fd-85718b01408d", "content-length": "728", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24592025-10-03T21:16:13.593ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
24602025-10-03T21:16:13.593ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6a5a284-8b81-4eb4-a141-d67697ef2291", "content-length": "722", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24612025-10-03T21:16:13.593ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
24622025-10-03T21:16:13.593ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "595794f7-eec9-4b17-b682-28f352442f00", "content-length": "10", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24632025-10-03T21:16:13.593ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
24642025-10-03T21:16:13.594ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd6773db-f601-43f2-9738-ab028ef7ee2a", "content-length": "144", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24652025-10-03T21:16:13.594ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
24662025-10-03T21:16:13.594ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62e3de24-89f8-46a4-8da3-ff440eccc8b9", "content-length": "147", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24672025-10-03T21:16:13.594ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
24682025-10-03T21:16:13.594ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e936144-33be-49d6-a651-2078cb990a03", "content-length": "109", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24692025-10-03T21:16:13.594ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
24702025-10-03T21:16:13.595ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6942171f-d70c-487e-abcf-4d31069d3456", "content-length": "109", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24712025-10-03T21:16:13.595ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
24722025-10-03T21:16:13.595ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf1793b-b809-482d-a51b-5bd53e178327", "content-length": "163", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24732025-10-03T21:16:13.595ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
24742025-10-03T21:16:13.595ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ad599be-d950-4f41-ae80-1eba1ed2e7c8", "content-length": "163", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24752025-10-03T21:16:13.595ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
24762025-10-03T21:16:13.596ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5473afa6-8c2d-4425-a116-fd4972342862", "content-length": "166", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24772025-10-03T21:16:13.596ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
24782025-10-03T21:16:13.596ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0c01c82-ba7b-4b28-b62f-3fe3b62ac59f", "content-length": "166", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24792025-10-03T21:16:13.596ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
24802025-10-03T21:16:13.596ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a797f0a2-1d01-453c-b747-c2bdf1c1533a", "content-length": "702", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24812025-10-03T21:16:13.596ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
24822025-10-03T21:16:13.596ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae9302b0-a162-4189-8cf5-202025cea6b0", "content-length": "727", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24832025-10-03T21:16:13.597ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
24842025-10-03T21:16:13.597ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cc80bb2-c6cd-4539-91cc-32ad599306ec", "content-length": "729", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24852025-10-03T21:16:13.597ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
24862025-10-03T21:16:13.597ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cb012fc-4967-45f4-8483-469be13dac90", "content-length": "728", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
24872025-10-03T21:16:13.597ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24882025-10-03T21:16:13.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24892025-10-03T21:16:13.599ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24902025-10-03T21:16:13.601ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24912025-10-03T21:16:13.601ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24922025-10-03T21:16:13.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24932025-10-03T21:16:13.603ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24942025-10-03T21:16:13.604ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24952025-10-03T21:16:13.604ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24962025-10-03T21:16:13.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24972025-10-03T21:16:13.637ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
24982025-10-03T21:16:13.637ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
24992025-10-03T21:16:13.694ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fcd91a8-c860-4bcd-bf6d-c2fa26e24e0b", "content-length": "88632", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25002025-10-03T21:16:13.696ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
25012025-10-03T21:16:13.696ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
25022025-10-03T21:16:13.696ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
25032025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
25042025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
25052025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
25062025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
25072025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
25082025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f34e3483-27db-4870-800a-ad9a252f51cf", "content-length": "64", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25092025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
25102025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da6e5274-d26d-4871-a6e7-3afb079803e3", "content-length": "64", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25112025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
25122025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6eafe9c8-8307-4f1f-81c3-703f616da2b7", "content-length": "64", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25132025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
25142025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "36c77063-8d54-45bd-80d5-5616fe8258c2", "content-length": "64", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25152025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
25162025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "56a62da4-779f-4eb8-8ba0-5049321f6765", "content-length": "64", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25172025-10-03T21:16:13.696ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
25182025-10-03T21:16:13.734ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9aa4d138-bf74-4938-9d65-3ef006baff4f", "content-length": "402011", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25192025-10-03T21:16:13.734ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "124763f3-4a8f-487d-adef-238e26b88932", "content-length": "401894", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25202025-10-03T21:16:13.735ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "87bc5eca-b316-449e-b2b3-a47f69e415bc", "content-length": "401927", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25212025-10-03T21:16:13.735ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "acf92c59-296d-4a3d-8b23-a35ed66c8286", "content-length": "394588", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25222025-10-03T21:16:13.736ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "12da005c-fb29-4ae5-a0be-37044039c537", "content-length": "428403", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25232025-10-03T21:16:13.739ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
25242025-10-03T21:16:13.739ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
25252025-10-03T21:16:13.739ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
25262025-10-03T21:16:13.739ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
25272025-10-03T21:16:13.739ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
25282025-10-03T21:16:13.773ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 (service)
25292025-10-03T21:16:13.773ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
25302025-10-03T21:16:13.777ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "48056841-21f9-41a6-b127-2cc82f2581da", "content-length": "122", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25312025-10-03T21:16:13.779ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
25322025-10-03T21:16:13.876ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
25332025-10-03T21:16:13.876ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
25342025-10-03T21:16:13.877ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f49c7468-a733-4a70-97aa-9640e43ba7dd", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25352025-10-03T21:16:13.877ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
25362025-10-03T21:16:13.879ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
25372025-10-03T21:16:13.879ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
25382025-10-03T21:16:13.880ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bfff11d-ab19-419c-99f1-86ff092c9e9c", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25392025-10-03T21:16:13.880ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
25402025-10-03T21:16:13.881ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
25412025-10-03T21:16:13.881ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
25422025-10-03T21:16:13.881ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5ff062d-409b-4b21-8964-c43787fb2b0c", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:16:13 GMT"} })
25432025-10-03T21:16:13.881ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
25442025-10-03T21:16:13.882ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
25452025-10-03T21:16:13.882ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
25462025-10-03T21:16:13.884ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
25472025-10-03T21:16:13.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
25482025-10-03T21:16:13.887ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25492025-10-03T21:16:13.889ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
25502025-10-03T21:16:13.889ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
25512025-10-03T21:16:13.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
25522025-10-03T21:16:14.286ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = bae9c0a0-bcad-48fe-9870-38aaaaabc637 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
25532025-10-03T21:16:14.286ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = bae9c0a0-bcad-48fe-9870-38aaaaabc637 time_started = 2025-10-03 21:16:13.589108 UTC
25542025-10-03T21:16:14.286ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Dependency
25552025-10-03T21:16:14.286ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 23 reason = Dependency
25562025-10-03T21:16:14.286ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 13 reason = Dependency
25572025-10-03T21:16:14.286ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25582025-10-03T21:16:14.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25592025-10-03T21:16:14.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
25602025-10-03T21:16:14.287ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 3.180263789s iteration = 11 status = {"collection_id":"bae9c0a0-bcad-48fe-9870-38aaaaabc637","time_done":"2025-10-03 21:16:13.882687 UTC","time_started":"2025-10-03 21:16:13.589108 UTC"}
25612025-10-03T21:16:14.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25622025-10-03T21:16:14.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
25632025-10-03T21:16:14.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25642025-10-03T21:16:14.288ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25652025-10-03T21:16:14.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25662025-10-03T21:16:14.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25672025-10-03T21:16:14.291ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25682025-10-03T21:16:14.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25692025-10-03T21:16:14.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
25702025-10-03T21:16:14.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
25712025-10-03T21:16:14.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
25722025-10-03T21:16:14.293ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25732025-10-03T21:16:14.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25742025-10-03T21:16:14.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25752025-10-03T21:16:14.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25762025-10-03T21:16:14.297ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25772025-10-03T21:16:14.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25782025-10-03T21:16:14.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
25792025-10-03T21:16:14.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
25802025-10-03T21:16:14.582ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 295.376954ms iteration = 30 status = {"physical_disks_added":0}
25812025-10-03T21:16:14.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25822025-10-03T21:16:14.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25832025-10-03T21:16:14.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25842025-10-03T21:16:14.622ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
25852025-10-03T21:16:14.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25862025-10-03T21:16:14.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25872025-10-03T21:16:14.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25882025-10-03T21:16:14.625ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
25892025-10-03T21:16:14.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25902025-10-03T21:16:14.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25912025-10-03T21:16:14.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25922025-10-03T21:16:14.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25932025-10-03T21:16:14.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25942025-10-03T21:16:14.631ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
25952025-10-03T21:16:14.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25962025-10-03T21:16:14.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25972025-10-03T21:16:14.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25982025-10-03T21:16:14.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
25992025-10-03T21:16:14.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26002025-10-03T21:16:14.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26012025-10-03T21:16:14.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26022025-10-03T21:16:14.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26032025-10-03T21:16:14.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26042025-10-03T21:16:14.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26052025-10-03T21:16:14.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26062025-10-03T21:16:14.643ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
26072025-10-03T21:16:14.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26082025-10-03T21:16:14.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26092025-10-03T21:16:14.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26102025-10-03T21:16:14.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26112025-10-03T21:16:14.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26122025-10-03T21:16:14.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
26132025-10-03T21:16:14.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26142025-10-03T21:16:14.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26152025-10-03T21:16:14.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26162025-10-03T21:16:14.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
26172025-10-03T21:16:14.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26182025-10-03T21:16:14.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26192025-10-03T21:16:14.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26202025-10-03T21:16:14.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26212025-10-03T21:16:14.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26222025-10-03T21:16:14.660ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
26232025-10-03T21:16:14.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26242025-10-03T21:16:14.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26252025-10-03T21:16:14.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26262025-10-03T21:16:14.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26272025-10-03T21:16:14.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26282025-10-03T21:16:14.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26292025-10-03T21:16:14.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26302025-10-03T21:16:14.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
26312025-10-03T21:16:14.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26322025-10-03T21:16:14.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26332025-10-03T21:16:14.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26342025-10-03T21:16:14.674ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
26352025-10-03T21:16:14.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26362025-10-03T21:16:14.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26372025-10-03T21:16:14.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26382025-10-03T21:16:14.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26392025-10-03T21:16:14.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26402025-10-03T21:16:14.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26412025-10-03T21:16:14.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26422025-10-03T21:16:14.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
26432025-10-03T21:16:14.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26442025-10-03T21:16:14.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26452025-10-03T21:16:14.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26462025-10-03T21:16:14.687ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
26472025-10-03T21:16:14.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26482025-10-03T21:16:14.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26492025-10-03T21:16:14.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26502025-10-03T21:16:14.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26512025-10-03T21:16:14.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26522025-10-03T21:16:14.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26532025-10-03T21:16:14.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26542025-10-03T21:16:14.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
26552025-10-03T21:16:14.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26562025-10-03T21:16:14.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26572025-10-03T21:16:14.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26582025-10-03T21:16:14.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26592025-10-03T21:16:14.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26602025-10-03T21:16:14.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
26612025-10-03T21:16:14.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26622025-10-03T21:16:14.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26632025-10-03T21:16:14.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26642025-10-03T21:16:14.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26652025-10-03T21:16:14.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26662025-10-03T21:16:14.708ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
26672025-10-03T21:16:14.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26682025-10-03T21:16:14.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26692025-10-03T21:16:14.710ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26702025-10-03T21:16:14.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26712025-10-03T21:16:14.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26722025-10-03T21:16:14.714ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
26732025-10-03T21:16:14.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26742025-10-03T21:16:14.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26752025-10-03T21:16:14.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26762025-10-03T21:16:14.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26772025-10-03T21:16:14.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26782025-10-03T21:16:14.721ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
26792025-10-03T21:16:14.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26802025-10-03T21:16:14.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26812025-10-03T21:16:14.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26822025-10-03T21:16:14.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26832025-10-03T21:16:14.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26842025-10-03T21:16:14.726ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
26852025-10-03T21:16:14.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26862025-10-03T21:16:14.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26872025-10-03T21:16:14.758ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
26882025-10-03T21:16:14.758ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
26892025-10-03T21:16:14.759ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = a96e92f9-c007-4734-8a92-4d63d987817b parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
26902025-10-03T21:16:14.759ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
26912025-10-03T21:16:14.759ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 472.182839ms iteration = 23 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
26922025-10-03T21:16:14.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
26932025-10-03T21:16:14.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
26942025-10-03T21:16:14.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26952025-10-03T21:16:14.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
26962025-10-03T21:16:14.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
26972025-10-03T21:16:14.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
26982025-10-03T21:16:14.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26992025-10-03T21:16:14.915ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
27002025-10-03T21:16:14.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
27012025-10-03T21:16:14.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
27022025-10-03T21:16:14.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
27032025-10-03T21:16:14.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
27042025-10-03T21:16:14.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27052025-10-03T21:16:14.920ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
27062025-10-03T21:16:14.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
27072025-10-03T21:16:14.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
27082025-10-03T21:16:14.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27092025-10-03T21:16:14.923ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
27102025-10-03T21:16:14.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
27112025-10-03T21:16:14.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
27122025-10-03T21:16:14.925ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
27132025-10-03T21:16:14.925ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 638.296044ms iteration = 13 status = {"blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","inventory_collection_id":"128c8459-d400-4e28-81e8-71ae30f5474f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
27142025-10-03T21:16:15.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 110 reason = Timeout
27152025-10-03T21:16:15.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27162025-10-03T21:16:15.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27172025-10-03T21:16:15.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27182025-10-03T21:16:15.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
27192025-10-03T21:16:15.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27202025-10-03T21:16:15.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27212025-10-03T21:16:15.196ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
27222025-10-03T21:16:15.196ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.549045ms iteration = 110 status = {"config_updated":false}
27232025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
27242025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
27252025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27262025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27272025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 111 reason = Timeout
27282025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27292025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27302025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27312025-10-03T21:16:20.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27322025-10-03T21:16:20.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27332025-10-03T21:16:20.192ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27342025-10-03T21:16:20.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27352025-10-03T21:16:20.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27362025-10-03T21:16:20.194ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.911794ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27372025-10-03T21:16:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
27382025-10-03T21:16:20.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27392025-10-03T21:16:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27402025-10-03T21:16:20.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27412025-10-03T21:16:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27422025-10-03T21:16:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27432025-10-03T21:16:20.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27442025-10-03T21:16:20.197ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
27452025-10-03T21:16:20.197ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.494964ms iteration = 111 status = {"config_updated":false}
27462025-10-03T21:16:20.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27472025-10-03T21:16:20.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27482025-10-03T21:16:20.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
27492025-10-03T21:16:20.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27502025-10-03T21:16:20.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27512025-10-03T21:16:20.214ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
27522025-10-03T21:16:20.214ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.966711ms iteration = 56 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
27532025-10-03T21:16:21.224ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 473 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 51c22e1e-fe90-4bd5-b6bb-c36850b9d249 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
27542025-10-03T21:16:25.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 112 reason = Timeout
27552025-10-03T21:16:25.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27562025-10-03T21:16:25.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27572025-10-03T21:16:25.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27582025-10-03T21:16:25.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
27592025-10-03T21:16:25.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27602025-10-03T21:16:25.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27612025-10-03T21:16:25.196ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
27622025-10-03T21:16:25.196ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.412402ms iteration = 112 status = {"config_updated":false}
27632025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
27642025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
27652025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27662025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 113 reason = Timeout
27672025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27682025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27692025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27702025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27712025-10-03T21:16:30.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27722025-10-03T21:16:30.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27732025-10-03T21:16:30.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27742025-10-03T21:16:30.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27752025-10-03T21:16:30.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27762025-10-03T21:16:30.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.312338ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27772025-10-03T21:16:30.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
27782025-10-03T21:16:30.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27792025-10-03T21:16:30.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27802025-10-03T21:16:30.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27812025-10-03T21:16:30.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27822025-10-03T21:16:30.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27832025-10-03T21:16:30.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
27842025-10-03T21:16:30.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.08231ms iteration = 113 status = {"config_updated":false}
27852025-10-03T21:16:30.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27862025-10-03T21:16:30.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27872025-10-03T21:16:30.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27882025-10-03T21:16:30.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
27892025-10-03T21:16:30.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27902025-10-03T21:16:30.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27912025-10-03T21:16:30.214ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
27922025-10-03T21:16:30.214ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.833129ms iteration = 57 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
27932025-10-03T21:16:31.223ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 399 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = bc259351-8b57-4915-8257-c52998fcdd35 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
27942025-10-03T21:16:35.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 114 reason = Timeout
27952025-10-03T21:16:35.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27962025-10-03T21:16:35.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27972025-10-03T21:16:35.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27982025-10-03T21:16:35.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
27992025-10-03T21:16:35.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
28002025-10-03T21:16:35.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
28012025-10-03T21:16:35.197ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
28022025-10-03T21:16:35.197ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.651245ms iteration = 114 status = {"config_updated":false}
28032025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
28042025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 20 reason = Timeout
28052025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
28062025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
28072025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
28082025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 20 reason = Timeout
28092025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.16\u{b5}s iteration = 20 status = {"disabled":true,"errors":[],"sps":[]}
28102025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
28112025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
28122025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
28132025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28142025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
28152025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
28162025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28172025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
28182025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 115 reason = Timeout
28192025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28202025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
28212025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
28222025-10-03T21:16:40.190ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
28232025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
28242025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
28252025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28262025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28272025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28282025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
28292025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
28302025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28312025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28322025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
28332025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
28342025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
28352025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
28362025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
28372025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28382025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
28392025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
28402025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 30 reason = Timeout
28412025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28422025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
28432025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
28442025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
28452025-10-03T21:16:40.190ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28462025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
28472025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
28482025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
28492025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
28502025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
28512025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
28522025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28532025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
28542025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
28552025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28562025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
28572025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28582025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
28592025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
28602025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28612025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28622025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
28632025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28642025-10-03T21:16:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28652025-10-03T21:16:40.194ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28662025-10-03T21:16:40.194ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28672025-10-03T21:16:40.194ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28682025-10-03T21:16:40.194ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28692025-10-03T21:16:40.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28702025-10-03T21:16:40.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28712025-10-03T21:16:40.195ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28722025-10-03T21:16:40.195ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28732025-10-03T21:16:40.195ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
28742025-10-03T21:16:40.195ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28752025-10-03T21:16:40.196ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.523032ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
28762025-10-03T21:16:40.196ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5045, tv_nsec: 685290236 } })
28772025-10-03T21:16:40.196ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5045, tv_nsec: 685675290 } })
28782025-10-03T21:16:40.197ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
28792025-10-03T21:16:40.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
28802025-10-03T21:16:40.197ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
28812025-10-03T21:16:40.198ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.928229ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
28822025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
28832025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28842025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28852025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28862025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
28872025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28882025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
28892025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28902025-10-03T21:16:40.200ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28912025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
28922025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28932025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28942025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28952025-10-03T21:16:40.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28962025-10-03T21:16:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
28972025-10-03T21:16:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28982025-10-03T21:16:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28992025-10-03T21:16:40.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29002025-10-03T21:16:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
29012025-10-03T21:16:40.201ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29022025-10-03T21:16:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
29032025-10-03T21:16:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
29042025-10-03T21:16:40.202ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
29052025-10-03T21:16:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29062025-10-03T21:16:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
29072025-10-03T21:16:40.204ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29082025-10-03T21:16:40.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29092025-10-03T21:16:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29102025-10-03T21:16:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29112025-10-03T21:16:40.207ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29122025-10-03T21:16:40.208ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29132025-10-03T21:16:40.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29142025-10-03T21:16:40.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29152025-10-03T21:16:40.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29162025-10-03T21:16:40.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
29172025-10-03T21:16:40.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29182025-10-03T21:16:40.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29192025-10-03T21:16:40.215ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29202025-10-03T21:16:40.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29212025-10-03T21:16:40.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29222025-10-03T21:16:40.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29232025-10-03T21:16:40.215ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29242025-10-03T21:16:40.215ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
29252025-10-03T21:16:40.215ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 25.309146ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
29262025-10-03T21:16:40.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
29272025-10-03T21:16:40.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 26.491949ms iteration = 115 status = {"config_updated":false}
29282025-10-03T21:16:40.224ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29292025-10-03T21:16:40.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29302025-10-03T21:16:40.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29312025-10-03T21:16:40.227ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29322025-10-03T21:16:40.229ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
29332025-10-03T21:16:40.229ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 38.824688ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
29342025-10-03T21:16:40.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
29352025-10-03T21:16:40.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29362025-10-03T21:16:40.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
29372025-10-03T21:16:40.231ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29382025-10-03T21:16:40.231ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29392025-10-03T21:16:40.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29402025-10-03T21:16:40.241ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 50.384599ms iteration = 31 status = {"physical_disks_added":0}
29412025-10-03T21:16:40.257ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
29422025-10-03T21:16:40.257ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.23823ms iteration = 58 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
29432025-10-03T21:16:40.267ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29442025-10-03T21:16:40.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29452025-10-03T21:16:40.270ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29462025-10-03T21:16:40.270ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29472025-10-03T21:16:40.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29482025-10-03T21:16:40.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
29492025-10-03T21:16:40.273ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29502025-10-03T21:16:40.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29512025-10-03T21:16:40.275ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 85.07054ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
29522025-10-03T21:16:40.275ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29532025-10-03T21:16:40.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29542025-10-03T21:16:40.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29552025-10-03T21:16:40.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29562025-10-03T21:16:40.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29572025-10-03T21:16:40.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29582025-10-03T21:16:40.277ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5045, tv_nsec: 766085008 } })
29592025-10-03T21:16:40.277ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 86.581658ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
29602025-10-03T21:16:40.277ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
29612025-10-03T21:16:40.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29622025-10-03T21:16:40.278ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 87.183194ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29632025-10-03T21:16:40.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 89.166676ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
29642025-10-03T21:16:40.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
29652025-10-03T21:16:40.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29662025-10-03T21:16:40.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29672025-10-03T21:16:40.284ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
29682025-10-03T21:16:40.284ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 93.218413ms iteration = 30 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
29692025-10-03T21:16:40.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 93.789298ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
29702025-10-03T21:16:40.292ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29712025-10-03T21:16:40.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
29722025-10-03T21:16:40.296ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 105.755394ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
29732025-10-03T21:16:40.299ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29742025-10-03T21:16:40.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29752025-10-03T21:16:40.301ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29762025-10-03T21:16:40.301ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29772025-10-03T21:16:40.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29782025-10-03T21:16:40.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29792025-10-03T21:16:40.303ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29802025-10-03T21:16:40.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29812025-10-03T21:16:40.304ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29822025-10-03T21:16:40.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
29832025-10-03T21:16:40.308ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29842025-10-03T21:16:40.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29852025-10-03T21:16:40.310ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 120.015355ms iteration = 20 status = 0
29862025-10-03T21:16:40.355ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
29872025-10-03T21:16:40.355ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29882025-10-03T21:16:40.356ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "693a899c-affa-4fb0-bd01-3d7f9f149e07", "content-length": "26", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
29892025-10-03T21:16:40.356ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29902025-10-03T21:16:40.358ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29912025-10-03T21:16:40.379ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
29922025-10-03T21:16:40.379ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29932025-10-03T21:16:40.379ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4300151-47a1-452a-b77e-f35d1b8c72b8", "content-length": "26", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
29942025-10-03T21:16:40.379ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29952025-10-03T21:16:40.381ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29962025-10-03T21:16:40.382ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
29972025-10-03T21:16:40.382ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29982025-10-03T21:16:40.382ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cc9032d-9b80-431d-bbc4-10ae0a506db5", "content-length": "26", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
29992025-10-03T21:16:40.382ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30002025-10-03T21:16:40.384ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
30012025-10-03T21:16:40.425ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30022025-10-03T21:16:40.425ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a1bace5-75aa-46d5-b324-dc49185aa517", "content-length": "691", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
30032025-10-03T21:16:40.425ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
30042025-10-03T21:16:40.425ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
30052025-10-03T21:16:40.425ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
30062025-10-03T21:16:40.427ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 236.566061ms iteration = 20 status = {}
30072025-10-03T21:16:40.456ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
30082025-10-03T21:16:40.456ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30092025-10-03T21:16:40.456ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b825cd5-e878-4bda-920f-ee9c50596b13", "content-length": "372", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
30102025-10-03T21:16:40.456ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30112025-10-03T21:16:40.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30122025-10-03T21:16:40.462ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 272.317934ms iteration = 20 status = {}
30132025-10-03T21:16:40.469ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
30142025-10-03T21:16:40.469ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
30152025-10-03T21:16:40.469ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5b48361f-ac45-46cf-8fc8-d6bf2fe52aa8", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
30162025-10-03T21:16:40.469ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30172025-10-03T21:16:40.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30182025-10-03T21:16:40.471ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
30192025-10-03T21:16:40.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
30202025-10-03T21:16:40.472ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30212025-10-03T21:16:40.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30222025-10-03T21:16:40.474ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 284.027476ms iteration = 20 status = null
30232025-10-03T21:16:40.475ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30242025-10-03T21:16:40.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30252025-10-03T21:16:40.479ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
30262025-10-03T21:16:40.479ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30272025-10-03T21:16:40.480ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c390d849-37d7-4d94-af31-c35aa32be5e7", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
30282025-10-03T21:16:40.485ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 294.400693ms iteration = 20 status = {}
30292025-10-03T21:16:40.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30302025-10-03T21:16:40.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30312025-10-03T21:16:40.543ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
30322025-10-03T21:16:40.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30332025-10-03T21:16:40.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30342025-10-03T21:16:40.558ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
30352025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
30362025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
30372025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
30382025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30392025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30402025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30412025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30422025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30432025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30442025-10-03T21:16:40.559ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
30452025-10-03T21:16:40.559ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30462025-10-03T21:16:40.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30472025-10-03T21:16:40.562ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
30482025-10-03T21:16:40.562ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
30492025-10-03T21:16:40.863ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7d9069b0-0e9f-4969-af88-26fca990da75", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
30502025-10-03T21:16:40.863ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
30512025-10-03T21:16:40.863ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
30522025-10-03T21:16:40.863ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ebad559-2128-4dc5-8829-6fdaf88134f6", "content-length": "682", "date": "Fri, 03 Oct 2025 21:16:40 GMT"} })
30532025-10-03T21:16:40.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30542025-10-03T21:16:40.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30552025-10-03T21:16:40.867ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30562025-10-03T21:16:40.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30572025-10-03T21:16:40.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30582025-10-03T21:16:40.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30592025-10-03T21:16:40.872ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30602025-10-03T21:16:40.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30612025-10-03T21:16:40.880ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 689.873287ms iteration = 20 status = {}
30622025-10-03T21:16:41.224ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 434 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 5bb19a0e-c1e2-4a57-9264-58a45f257327 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
30632025-10-03T21:16:45.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 116 reason = Timeout
30642025-10-03T21:16:45.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
30652025-10-03T21:16:45.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
30662025-10-03T21:16:45.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30672025-10-03T21:16:45.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
30682025-10-03T21:16:45.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
30692025-10-03T21:16:45.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
30702025-10-03T21:16:45.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
30712025-10-03T21:16:45.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.816748ms iteration = 116 status = {"config_updated":false}
30722025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
30732025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
30742025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30752025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30762025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 117 reason = Timeout
30772025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
30782025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
30792025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30802025-10-03T21:16:50.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
30812025-10-03T21:16:50.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30822025-10-03T21:16:50.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30832025-10-03T21:16:50.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30842025-10-03T21:16:50.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30852025-10-03T21:16:50.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.677042ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30862025-10-03T21:16:50.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30872025-10-03T21:16:50.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30882025-10-03T21:16:50.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
30892025-10-03T21:16:50.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
30902025-10-03T21:16:50.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
30912025-10-03T21:16:50.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
30922025-10-03T21:16:50.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
30932025-10-03T21:16:50.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.206371ms iteration = 117 status = {"config_updated":false}
30942025-10-03T21:16:50.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30952025-10-03T21:16:50.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
30962025-10-03T21:16:50.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30972025-10-03T21:16:50.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
30982025-10-03T21:16:50.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30992025-10-03T21:16:50.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31002025-10-03T21:16:50.219ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
31012025-10-03T21:16:50.219ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.790894ms iteration = 59 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
31022025-10-03T21:16:51.224ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 415 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = f659eb86-9cdb-4d3d-910c-5bf20f3862eb response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
31032025-10-03T21:16:55.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 118 reason = Timeout
31042025-10-03T21:16:55.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31052025-10-03T21:16:55.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31062025-10-03T21:16:55.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31072025-10-03T21:16:55.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
31082025-10-03T21:16:55.200ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31092025-10-03T21:16:55.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31102025-10-03T21:16:55.202ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
31112025-10-03T21:16:55.202ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 10.779232ms iteration = 118 status = {"config_updated":false}
31122025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
31132025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31142025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
31152025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31162025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 119 reason = Timeout
31172025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31182025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31192025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31202025-10-03T21:17:00.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31212025-10-03T21:17:00.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31222025-10-03T21:17:00.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31232025-10-03T21:17:00.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31242025-10-03T21:17:00.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31252025-10-03T21:17:00.194ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.267257ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31262025-10-03T21:17:00.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31272025-10-03T21:17:00.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31282025-10-03T21:17:00.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31292025-10-03T21:17:00.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
31302025-10-03T21:17:00.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31312025-10-03T21:17:00.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31322025-10-03T21:17:00.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
31332025-10-03T21:17:00.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.865468ms iteration = 119 status = {"config_updated":false}
31342025-10-03T21:17:00.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31352025-10-03T21:17:00.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31362025-10-03T21:17:00.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31372025-10-03T21:17:00.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
31382025-10-03T21:17:00.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31392025-10-03T21:17:00.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31402025-10-03T21:17:00.214ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
31412025-10-03T21:17:00.214ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.449565ms iteration = 60 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
31422025-10-03T21:17:01.225ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 590 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 1fcd6ee0-82be-473b-b065-260d7d2b193c response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
31432025-10-03T21:17:05.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 120 reason = Timeout
31442025-10-03T21:17:05.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31452025-10-03T21:17:05.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31462025-10-03T21:17:05.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31472025-10-03T21:17:05.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
31482025-10-03T21:17:05.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31492025-10-03T21:17:05.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31502025-10-03T21:17:05.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
31512025-10-03T21:17:05.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.485743ms iteration = 120 status = {"config_updated":false}
31522025-10-03T21:17:06.217ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:37452
31532025-10-03T21:17:06.218ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 9b8cf59e-3685-460f-a40b-719cd7091878 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
31542025-10-03T21:17:06.218ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 9b8cf59e-3685-460f-a40b-719cd7091878 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
31552025-10-03T21:17:06.221ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3150 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 9b8cf59e-3685-460f-a40b-719cd7091878 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending
31562025-10-03T21:17:06.221ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 0596ae20-6973-4976-b043-d0a9c3eee495 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
31572025-10-03T21:17:06.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 0596ae20-6973-4976-b043-d0a9c3eee495 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
31582025-10-03T21:17:06.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2467 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 0596ae20-6973-4976-b043-d0a9c3eee495 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D
31592025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
31602025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
31612025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
31622025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
31632025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
31642025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 21 reason = Timeout
31652025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
31662025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
31672025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
31682025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
31692025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
31702025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31712025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
31722025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31732025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
31742025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
31752025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31762025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
31772025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
31782025-10-03T21:17:10.190ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
31792025-10-03T21:17:10.190ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
31802025-10-03T21:17:10.190ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-10-03 21:07:10.190923278 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
31812025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
31822025-10-03T21:17:10.191ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = ExternalNetworking step id = Ensure
31832025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31842025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
31852025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
31862025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
31872025-10-03T21:17:10.191ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:510
31882025-10-03T21:17:10.191ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
31892025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
31902025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
31912025-10-03T21:17:10.191ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
31922025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31932025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
31942025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31952025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
31962025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31972025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
31982025-10-03T21:17:10.191ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
31992025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32002025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
32012025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
32022025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32032025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32042025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 121 reason = Timeout
32052025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
32062025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
32072025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
32082025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
32092025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
32102025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
32112025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
32122025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
32132025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
32142025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
32152025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32162025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
32172025-10-03T21:17:10.191ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
32182025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 31 reason = Timeout
32192025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
32202025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32212025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
32222025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
32232025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
32242025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 24 reason = Timeout
32252025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32262025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
32272025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
32282025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 14 reason = Timeout
32292025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 40.771\u{b5}s iteration = 21 status = {"disabled":true,"errors":[],"sps":[]}
32302025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
32312025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
32322025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
32332025-10-03T21:17:10.191ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
32342025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
32352025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
32362025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
32372025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
32382025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
32392025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
32402025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
32412025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32422025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
32432025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
32442025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32452025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32462025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
32472025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
32482025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
32492025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32502025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
32512025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
32522025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
32532025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32542025-10-03T21:17:10.191ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a (built_in_user), .. }) resource = Database result = Ok(())
32552025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
32562025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 12 reason = Timeout
32572025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 3 reason = Timeout
32582025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32592025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
32602025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
32612025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32622025-10-03T21:17:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
32632025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
32642025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
32652025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
32662025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
32672025-10-03T21:17:10.192ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("486849a9-1733-4aa9-9b66-43ace2da0eea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 5075, tv_nsec: 680480941 } })
32682025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
32692025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32702025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
32712025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32722025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32732025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
32742025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
32752025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
32762025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
32772025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32782025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32792025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
32802025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
32812025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32822025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
32832025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
32842025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
32852025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
32862025-10-03T21:17:10.192ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2efcfb05-0c3d-45aa-b120-c334f3706be4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a7c1656-d513-4cc3-b2b0-8e31ad2b5545.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 5075, tv_nsec: 680813875 } })
32872025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32882025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
32892025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
32902025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
32912025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
32922025-10-03T21:17:10.192ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.091424ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
32932025-10-03T21:17:10.193ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
32942025-10-03T21:17:10.193ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.146604ms iteration = 11 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
32952025-10-03T21:17:10.195ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32962025-10-03T21:17:10.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.438098ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
32972025-10-03T21:17:10.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.52031ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
32982025-10-03T21:17:10.195ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32992025-10-03T21:17:10.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33002025-10-03T21:17:10.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33012025-10-03T21:17:10.196ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33022025-10-03T21:17:10.197ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
33032025-10-03T21:17:10.197ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 5.592243ms iteration = 11 status = {"by_rx":{},"error":null}
33042025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.602067ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
33052025-10-03T21:17:10.199ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
33062025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
33072025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33082025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33092025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
33102025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
33112025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
33122025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33132025-10-03T21:17:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
33142025-10-03T21:17:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
33152025-10-03T21:17:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33162025-10-03T21:17:10.200ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33172025-10-03T21:17:10.200ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33182025-10-03T21:17:10.200ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
33192025-10-03T21:17:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33202025-10-03T21:17:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33212025-10-03T21:17:10.201ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33222025-10-03T21:17:10.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33232025-10-03T21:17:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
33242025-10-03T21:17:10.201ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.642271ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
33252025-10-03T21:17:10.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
33262025-10-03T21:17:10.202ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
33272025-10-03T21:17:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
33282025-10-03T21:17:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33292025-10-03T21:17:10.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33302025-10-03T21:17:10.202ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5075, tv_nsec: 691091911 } })
33312025-10-03T21:17:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
33322025-10-03T21:17:10.203ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
33332025-10-03T21:17:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
33342025-10-03T21:17:10.203ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
33352025-10-03T21:17:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
33362025-10-03T21:17:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
33372025-10-03T21:17:10.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33382025-10-03T21:17:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33392025-10-03T21:17:10.204ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5075, tv_nsec: 692856981 } })
33402025-10-03T21:17:10.204ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33412025-10-03T21:17:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33422025-10-03T21:17:10.205ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
33432025-10-03T21:17:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
33442025-10-03T21:17:10.205ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
33452025-10-03T21:17:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
33462025-10-03T21:17:10.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
33472025-10-03T21:17:10.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
33482025-10-03T21:17:10.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
33492025-10-03T21:17:10.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33502025-10-03T21:17:10.210ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33512025-10-03T21:17:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
33522025-10-03T21:17:10.211ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33532025-10-03T21:17:10.211ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33542025-10-03T21:17:10.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
33552025-10-03T21:17:10.213ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33562025-10-03T21:17:10.214ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
33572025-10-03T21:17:10.214ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33582025-10-03T21:17:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
33592025-10-03T21:17:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
33602025-10-03T21:17:10.216ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
33612025-10-03T21:17:10.216ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
33622025-10-03T21:17:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
33632025-10-03T21:17:10.218ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33642025-10-03T21:17:10.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33652025-10-03T21:17:10.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
33662025-10-03T21:17:10.222ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33672025-10-03T21:17:10.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33682025-10-03T21:17:10.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
33692025-10-03T21:17:10.224ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33702025-10-03T21:17:10.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33712025-10-03T21:17:10.227ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33722025-10-03T21:17:10.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33732025-10-03T21:17:10.229ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33742025-10-03T21:17:10.229ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-10-03 21:07:10.190923278 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
33752025-10-03T21:17:10.229ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 38.97133ms iteration = 11 status = {"expiration":"2025-10-03T21:07:10.190923278Z","pruned":[]}
33762025-10-03T21:17:10.229ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33772025-10-03T21:17:10.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
33782025-10-03T21:17:10.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
33792025-10-03T21:17:10.237ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
33802025-10-03T21:17:10.237ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 46.462205ms iteration = 121 status = {"config_updated":false}
33812025-10-03T21:17:10.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33822025-10-03T21:17:10.239ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
33832025-10-03T21:17:10.239ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
33842025-10-03T21:17:10.239ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
33852025-10-03T21:17:10.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33862025-10-03T21:17:10.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
33872025-10-03T21:17:10.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
33882025-10-03T21:17:10.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
33892025-10-03T21:17:10.242ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
33902025-10-03T21:17:10.242ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
33912025-10-03T21:17:10.242ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
33922025-10-03T21:17:10.242ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33932025-10-03T21:17:10.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33942025-10-03T21:17:10.243ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
33952025-10-03T21:17:10.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
33962025-10-03T21:17:10.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33972025-10-03T21:17:10.243ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 52.792296ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
33982025-10-03T21:17:10.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33992025-10-03T21:17:10.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34002025-10-03T21:17:10.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34012025-10-03T21:17:10.246ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34022025-10-03T21:17:10.246ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 55.035351ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
34032025-10-03T21:17:10.248ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
34042025-10-03T21:17:10.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
34052025-10-03T21:17:10.251ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34062025-10-03T21:17:10.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
34072025-10-03T21:17:10.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
34082025-10-03T21:17:10.252ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34092025-10-03T21:17:10.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34102025-10-03T21:17:10.255ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34112025-10-03T21:17:10.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34122025-10-03T21:17:10.257ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5075, tv_nsec: 746070322 } })
34132025-10-03T21:17:10.258ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
34142025-10-03T21:17:10.259ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34152025-10-03T21:17:10.263ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
34162025-10-03T21:17:10.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34172025-10-03T21:17:10.263ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34182025-10-03T21:17:10.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34192025-10-03T21:17:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
34202025-10-03T21:17:10.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34212025-10-03T21:17:10.267ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34222025-10-03T21:17:10.267ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
34232025-10-03T21:17:10.267ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 76.907708ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
34242025-10-03T21:17:10.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
34252025-10-03T21:17:10.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34262025-10-03T21:17:10.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
34272025-10-03T21:17:10.270ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34282025-10-03T21:17:10.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34292025-10-03T21:17:10.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34302025-10-03T21:17:10.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
34312025-10-03T21:17:10.273ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
34322025-10-03T21:17:10.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
34332025-10-03T21:17:10.279ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 88.587509ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"195.0.0","globs_reprocessed":{},"no_receivers":[]}
34342025-10-03T21:17:10.294ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
34352025-10-03T21:17:10.294ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
34362025-10-03T21:17:10.294ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 103.248206ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
34372025-10-03T21:17:10.298ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34382025-10-03T21:17:10.302ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34392025-10-03T21:17:10.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34402025-10-03T21:17:10.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
34412025-10-03T21:17:10.304ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34422025-10-03T21:17:10.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
34432025-10-03T21:17:10.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34442025-10-03T21:17:10.312ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
34452025-10-03T21:17:10.312ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
34462025-10-03T21:17:10.312ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
34472025-10-03T21:17:10.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34482025-10-03T21:17:10.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34492025-10-03T21:17:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34502025-10-03T21:17:10.315ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
34512025-10-03T21:17:10.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
34522025-10-03T21:17:10.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34532025-10-03T21:17:10.322ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
34542025-10-03T21:17:10.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
34552025-10-03T21:17:10.323ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34562025-10-03T21:17:10.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34572025-10-03T21:17:10.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
34582025-10-03T21:17:10.324ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34592025-10-03T21:17:10.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34602025-10-03T21:17:10.325ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 133.82197ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
34612025-10-03T21:17:10.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34622025-10-03T21:17:10.327ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
34632025-10-03T21:17:10.329ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34642025-10-03T21:17:10.330ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
34652025-10-03T21:17:10.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
34662025-10-03T21:17:10.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34672025-10-03T21:17:10.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34682025-10-03T21:17:10.332ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
34692025-10-03T21:17:10.333ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34702025-10-03T21:17:10.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
34712025-10-03T21:17:10.333ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
34722025-10-03T21:17:10.334ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34732025-10-03T21:17:10.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
34742025-10-03T21:17:10.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34752025-10-03T21:17:10.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34762025-10-03T21:17:10.337ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34772025-10-03T21:17:10.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34782025-10-03T21:17:10.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34792025-10-03T21:17:10.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34802025-10-03T21:17:10.338ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
34812025-10-03T21:17:10.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
34822025-10-03T21:17:10.338ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
34832025-10-03T21:17:10.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
34842025-10-03T21:17:10.340ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34852025-10-03T21:17:10.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34862025-10-03T21:17:10.342ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34872025-10-03T21:17:10.342ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
34882025-10-03T21:17:10.342ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34892025-10-03T21:17:10.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34902025-10-03T21:17:10.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
34912025-10-03T21:17:10.344ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34922025-10-03T21:17:10.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
34932025-10-03T21:17:10.344ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34942025-10-03T21:17:10.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34952025-10-03T21:17:10.345ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
34962025-10-03T21:17:10.345ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
34972025-10-03T21:17:10.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34982025-10-03T21:17:10.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34992025-10-03T21:17:10.346ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:114 status = TufRepoPrunerStatus { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
35002025-10-03T21:17:10.346ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 154.226301ms iteration = 3 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"warnings":[]}
35012025-10-03T21:17:10.346ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
35022025-10-03T21:17:10.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35032025-10-03T21:17:10.347ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35042025-10-03T21:17:10.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35052025-10-03T21:17:10.350ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35062025-10-03T21:17:10.352ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35072025-10-03T21:17:10.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
35082025-10-03T21:17:10.353ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
35092025-10-03T21:17:10.353ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
35102025-10-03T21:17:10.353ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished blueprint_id = Some(675bea75-8e2d-4986-86f2-b590a30bb084 (blueprint)) file = nexus/types/src/quiesce.rs:525 generation = 1
35112025-10-03T21:17:10.353ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 162.775748ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
35122025-10-03T21:17:10.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
35132025-10-03T21:17:10.354ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 163.247413ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35142025-10-03T21:17:10.355ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
35152025-10-03T21:17:10.357ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
35162025-10-03T21:17:10.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
35172025-10-03T21:17:10.358ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35182025-10-03T21:17:10.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35192025-10-03T21:17:10.361ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
35202025-10-03T21:17:10.363ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
35212025-10-03T21:17:10.363ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
35222025-10-03T21:17:10.363ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 172.605648ms iteration = 61 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
35232025-10-03T21:17:10.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35242025-10-03T21:17:10.365ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
35252025-10-03T21:17:10.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35262025-10-03T21:17:10.366ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35272025-10-03T21:17:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35282025-10-03T21:17:10.367ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
35292025-10-03T21:17:10.367ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 175.620623ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
35302025-10-03T21:17:10.367ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
35312025-10-03T21:17:10.368ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = SledList step id = Fetch
35322025-10-03T21:17:10.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
35332025-10-03T21:17:10.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35342025-10-03T21:17:10.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
35352025-10-03T21:17:10.369ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35362025-10-03T21:17:10.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35372025-10-03T21:17:10.372ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35382025-10-03T21:17:10.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35392025-10-03T21:17:10.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35402025-10-03T21:17:10.372ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35412025-10-03T21:17:10.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35422025-10-03T21:17:10.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
35432025-10-03T21:17:10.379ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35442025-10-03T21:17:10.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35452025-10-03T21:17:10.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35462025-10-03T21:17:10.381ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35472025-10-03T21:17:10.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35482025-10-03T21:17:10.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35492025-10-03T21:17:10.384ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
35502025-10-03T21:17:10.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
35512025-10-03T21:17:10.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
35522025-10-03T21:17:10.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35532025-10-03T21:17:10.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35542025-10-03T21:17:10.388ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35552025-10-03T21:17:10.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35562025-10-03T21:17:10.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35572025-10-03T21:17:10.396ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35582025-10-03T21:17:10.396ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35592025-10-03T21:17:10.397ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
35602025-10-03T21:17:10.397ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
35612025-10-03T21:17:10.397ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35622025-10-03T21:17:10.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35632025-10-03T21:17:10.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35642025-10-03T21:17:10.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35652025-10-03T21:17:10.398ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35662025-10-03T21:17:10.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35672025-10-03T21:17:10.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35682025-10-03T21:17:10.400ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35692025-10-03T21:17:10.400ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35702025-10-03T21:17:10.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35712025-10-03T21:17:10.401ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35722025-10-03T21:17:10.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35732025-10-03T21:17:10.402ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35742025-10-03T21:17:10.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35752025-10-03T21:17:10.405ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35762025-10-03T21:17:10.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35772025-10-03T21:17:10.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35782025-10-03T21:17:10.405ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 214.190267ms iteration = 32 status = {"physical_disks_added":0}
35792025-10-03T21:17:10.407ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
35802025-10-03T21:17:10.407ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
35812025-10-03T21:17:10.407ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35822025-10-03T21:17:10.407ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35832025-10-03T21:17:10.407ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35842025-10-03T21:17:10.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35852025-10-03T21:17:10.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35862025-10-03T21:17:10.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35872025-10-03T21:17:10.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
35882025-10-03T21:17:10.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
35892025-10-03T21:17:10.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
35902025-10-03T21:17:10.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35912025-10-03T21:17:10.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35922025-10-03T21:17:10.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35932025-10-03T21:17:10.410ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
35942025-10-03T21:17:10.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
35952025-10-03T21:17:10.411ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 219.327266ms iteration = 21 status = 0
35962025-10-03T21:17:10.412ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35972025-10-03T21:17:10.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
35982025-10-03T21:17:10.413ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
35992025-10-03T21:17:10.413ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
36002025-10-03T21:17:10.413ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-10-03 21:07:12.445842 UTC dns_group = internal generation = 1
36012025-10-03T21:17:10.413ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 222.648304ms iteration = 11 status = {"generation":1}
36022025-10-03T21:17:10.414ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
36032025-10-03T21:17:10.414ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
36042025-10-03T21:17:10.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
36052025-10-03T21:17:10.416ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = DeployNexusRecords step id = Ensure
36062025-10-03T21:17:10.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
36072025-10-03T21:17:10.417ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36082025-10-03T21:17:10.417ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36092025-10-03T21:17:10.417ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
36102025-10-03T21:17:10.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
36112025-10-03T21:17:10.417ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 226.350475ms iteration = 14 status = {"blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","inventory_collection_id":"128c8459-d400-4e28-81e8-71ae30f5474f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
36122025-10-03T21:17:10.418ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36132025-10-03T21:17:10.421ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
36142025-10-03T21:17:10.421ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
36152025-10-03T21:17:10.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
36162025-10-03T21:17:10.422ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
36172025-10-03T21:17:10.422ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
36182025-10-03T21:17:10.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
36192025-10-03T21:17:10.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
36202025-10-03T21:17:10.425ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36212025-10-03T21:17:10.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36222025-10-03T21:17:10.434ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
36232025-10-03T21:17:10.434ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36242025-10-03T21:17:10.434ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
36252025-10-03T21:17:10.434ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36262025-10-03T21:17:10.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36272025-10-03T21:17:10.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36282025-10-03T21:17:10.437ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
36292025-10-03T21:17:10.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
36302025-10-03T21:17:10.437ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
36312025-10-03T21:17:10.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
36322025-10-03T21:17:10.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36332025-10-03T21:17:10.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
36342025-10-03T21:17:10.439ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36352025-10-03T21:17:10.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36362025-10-03T21:17:10.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
36372025-10-03T21:17:10.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
36382025-10-03T21:17:10.443ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36392025-10-03T21:17:10.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
36402025-10-03T21:17:10.443ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
36412025-10-03T21:17:10.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36422025-10-03T21:17:10.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
36432025-10-03T21:17:10.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
36442025-10-03T21:17:10.444ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36452025-10-03T21:17:10.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36462025-10-03T21:17:10.446ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
36472025-10-03T21:17:10.446ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
36482025-10-03T21:17:10.446ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 254.607984ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
36492025-10-03T21:17:10.454ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(8a0d3c05-6d9a-45d0-95fb-f2b68709aa25, 0), (e33257de-ae23-49ef-9065-e527da3bdcfa, 0), (bae9c0a0-bcad-48fe-9870-38aaaaabc637, 0), (aae16669-bf15-4c64-abd0-29f252338280, 0)] collection_id = 8a0d3c05-6d9a-45d0-95fb-f2b68709aa25
36502025-10-03T21:17:10.454ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36512025-10-03T21:17:10.454ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36522025-10-03T21:17:10.454ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
36532025-10-03T21:17:10.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
36542025-10-03T21:17:10.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36552025-10-03T21:17:10.458ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "e118fb37-0dec-48bc-a2df-7bd816b2b69e", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
36562025-10-03T21:17:10.458ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
36572025-10-03T21:17:10.458ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "51934801-46a3-4c45-aa89-cbabb39111e5", "content-length": "26", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
36582025-10-03T21:17:10.458ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:673 sled = http://[fd00:1122:3344:101::1]:12345
36592025-10-03T21:17:10.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36602025-10-03T21:17:10.460ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36612025-10-03T21:17:10.460ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 269.483332ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
36622025-10-03T21:17:10.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36632025-10-03T21:17:10.461ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
36642025-10-03T21:17:10.461ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
36652025-10-03T21:17:10.461ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-10-03 21:07:14.470425 UTC dns_group = external generation = 2
36662025-10-03T21:17:10.461ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 270.497214ms iteration = 11 status = {"generation":2}
36672025-10-03T21:17:10.461ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 270.348982ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"87c00105-6187-43fb-ab98-0bf5dda28bd9","time":"2025-10-03T21:07:10.861279585Z"},{"operation":"list","target_sled":"87c00105-6187-43fb-ab98-0bf5dda28bd9","time":"2025-10-03T21:07:10.861520488Z"},{"generation":1,"operation":"put_config","target_sled":"87c00105-6187-43fb-ab98-0bf5dda28bd9","time":"2025-10-03T21:12:10.724391691Z"},{"operation":"list","target_sled":"87c00105-6187-43fb-ab98-0bf5dda28bd9","time":"2025-10-03T21:12:10.724635123Z"},{"generation":1,"operation":"put_config","target_sled":"87c00105-6187-43fb-ab98-0bf5dda28bd9","time":"2025-10-03T21:17:10.458029124Z"},{"operation":"list","target_sled":"87c00105-6187-43fb-ab98-0bf5dda28bd9","time":"2025-10-03T21:17:10.458675391Z"}]}
36682025-10-03T21:17:10.462ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
36692025-10-03T21:17:10.462ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36702025-10-03T21:17:10.463ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7707af36-87ca-476d-97ca-b6d3f1a2ca3b", "content-length": "26", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
36712025-10-03T21:17:10.463ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36722025-10-03T21:17:10.463ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
36732025-10-03T21:17:10.463ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36742025-10-03T21:17:10.464ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "666872e0-0d3a-455b-956b-f68e38f78018", "content-length": "26", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
36752025-10-03T21:17:10.464ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36762025-10-03T21:17:10.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
36772025-10-03T21:17:10.465ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36782025-10-03T21:17:10.465ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36792025-10-03T21:17:10.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36802025-10-03T21:17:10.467ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
36812025-10-03T21:17:10.474ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = SledAgent step id = Ensure
36822025-10-03T21:17:10.477ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36832025-10-03T21:17:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36842025-10-03T21:17:10.484ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36852025-10-03T21:17:10.491ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
36862025-10-03T21:17:10.491ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36872025-10-03T21:17:10.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36882025-10-03T21:17:10.494ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36892025-10-03T21:17:10.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36902025-10-03T21:17:10.497ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
36912025-10-03T21:17:10.497ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 305.591759ms iteration = 31 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
36922025-10-03T21:17:10.497ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
36932025-10-03T21:17:10.498ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
36942025-10-03T21:17:10.498ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36952025-10-03T21:17:10.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
36962025-10-03T21:17:10.499ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe3cb66e-9aba-4252-b0f3-04c5ab383f7a", "content-length": "26", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
36972025-10-03T21:17:10.499ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36982025-10-03T21:17:10.501ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36992025-10-03T21:17:10.502ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37002025-10-03T21:17:10.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
37012025-10-03T21:17:10.507ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37022025-10-03T21:17:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37032025-10-03T21:17:10.514ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37042025-10-03T21:17:10.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37052025-10-03T21:17:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37062025-10-03T21:17:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37072025-10-03T21:17:10.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37082025-10-03T21:17:10.516ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37092025-10-03T21:17:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37102025-10-03T21:17:10.519ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37112025-10-03T21:17:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37122025-10-03T21:17:10.520ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
37132025-10-03T21:17:10.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37142025-10-03T21:17:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37152025-10-03T21:17:10.522ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37162025-10-03T21:17:10.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37172025-10-03T21:17:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37182025-10-03T21:17:10.524ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-10-03T21:07:04.185535Z, time_modified: 2025-10-03T21:07:04.185535Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }) }
37192025-10-03T21:17:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 334.338294ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]}},"default_endpoint":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
37202025-10-03T21:17:10.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
37212025-10-03T21:17:10.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37222025-10-03T21:17:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37232025-10-03T21:17:10.525ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37242025-10-03T21:17:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37252025-10-03T21:17:10.529ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37262025-10-03T21:17:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37272025-10-03T21:17:10.532ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37282025-10-03T21:17:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37292025-10-03T21:17:10.535ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37302025-10-03T21:17:10.539ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
37312025-10-03T21:17:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 347.691104ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
37322025-10-03T21:17:10.541ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
37332025-10-03T21:17:10.541ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37342025-10-03T21:17:10.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37352025-10-03T21:17:10.544ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37362025-10-03T21:17:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37372025-10-03T21:17:10.544ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
37382025-10-03T21:17:10.546ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37392025-10-03T21:17:10.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37402025-10-03T21:17:10.548ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37412025-10-03T21:17:10.549ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37422025-10-03T21:17:10.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
37432025-10-03T21:17:10.551ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37442025-10-03T21:17:10.555ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37452025-10-03T21:17:10.555ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37462025-10-03T21:17:10.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
37472025-10-03T21:17:10.557ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
37482025-10-03T21:17:10.557ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37492025-10-03T21:17:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37502025-10-03T21:17:10.558ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37512025-10-03T21:17:10.561ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37522025-10-03T21:17:10.561ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37532025-10-03T21:17:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
37542025-10-03T21:17:10.576ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
37552025-10-03T21:17:10.576ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37562025-10-03T21:17:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37572025-10-03T21:17:10.580ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37582025-10-03T21:17:10.583ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
37592025-10-03T21:17:10.583ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37602025-10-03T21:17:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
37612025-10-03T21:17:10.587ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
37622025-10-03T21:17:10.587ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
37632025-10-03T21:17:10.587ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
37642025-10-03T21:17:10.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37652025-10-03T21:17:10.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37662025-10-03T21:17:10.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37672025-10-03T21:17:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37682025-10-03T21:17:10.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37692025-10-03T21:17:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
37702025-10-03T21:17:10.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37712025-10-03T21:17:10.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37722025-10-03T21:17:10.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37732025-10-03T21:17:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37742025-10-03T21:17:10.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37752025-10-03T21:17:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
37762025-10-03T21:17:10.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37772025-10-03T21:17:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37782025-10-03T21:17:10.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37792025-10-03T21:17:10.618ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), time_created: 2025-10-03T21:07:05.994Z, time_modified: 2025-10-03T21:07:05.994Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
37802025-10-03T21:17:10.618ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
37812025-10-03T21:17:10.622ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
37822025-10-03T21:17:10.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37832025-10-03T21:17:10.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37842025-10-03T21:17:10.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37852025-10-03T21:17:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37862025-10-03T21:17:10.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37872025-10-03T21:17:10.632ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
37882025-10-03T21:17:10.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37892025-10-03T21:17:10.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37902025-10-03T21:17:10.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37912025-10-03T21:17:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37922025-10-03T21:17:10.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37932025-10-03T21:17:10.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37942025-10-03T21:17:10.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37952025-10-03T21:17:10.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
37962025-10-03T21:17:10.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37972025-10-03T21:17:10.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37982025-10-03T21:17:10.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37992025-10-03T21:17:10.656ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38002025-10-03T21:17:10.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38012025-10-03T21:17:10.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38022025-10-03T21:17:10.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38032025-10-03T21:17:10.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38042025-10-03T21:17:10.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38052025-10-03T21:17:10.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38062025-10-03T21:17:10.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38072025-10-03T21:17:10.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38082025-10-03T21:17:10.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38092025-10-03T21:17:10.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38102025-10-03T21:17:10.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38112025-10-03T21:17:10.675ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38122025-10-03T21:17:10.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38132025-10-03T21:17:10.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38142025-10-03T21:17:10.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38152025-10-03T21:17:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38162025-10-03T21:17:10.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38172025-10-03T21:17:10.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38182025-10-03T21:17:10.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38192025-10-03T21:17:10.685ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 494.985907ms iteration = 21 status = {}
38202025-10-03T21:17:10.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
38212025-10-03T21:17:10.714ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 8a0d3c05-6d9a-45d0-95fb-f2b68709aa25 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
38222025-10-03T21:17:10.714ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38232025-10-03T21:17:10.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
38242025-10-03T21:17:10.724ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
38252025-10-03T21:17:10.726ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "e78806bb-077a-41ee-87ba-279110194a96", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} }) sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
38262025-10-03T21:17:10.728ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = FirewallRules step id = Ensure
38272025-10-03T21:17:10.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38282025-10-03T21:17:10.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38292025-10-03T21:17:10.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38302025-10-03T21:17:10.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38312025-10-03T21:17:10.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38322025-10-03T21:17:10.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38332025-10-03T21:17:10.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38342025-10-03T21:17:10.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38352025-10-03T21:17:10.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38362025-10-03T21:17:10.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38372025-10-03T21:17:10.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38382025-10-03T21:17:10.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38392025-10-03T21:17:10.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38402025-10-03T21:17:10.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38412025-10-03T21:17:10.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38422025-10-03T21:17:10.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38432025-10-03T21:17:10.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38442025-10-03T21:17:10.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38452025-10-03T21:17:10.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
38462025-10-03T21:17:10.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38472025-10-03T21:17:10.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38482025-10-03T21:17:10.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38492025-10-03T21:17:10.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38502025-10-03T21:17:10.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
38512025-10-03T21:17:10.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38522025-10-03T21:17:10.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38532025-10-03T21:17:10.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38542025-10-03T21:17:10.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38552025-10-03T21:17:10.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38562025-10-03T21:17:10.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38572025-10-03T21:17:10.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38582025-10-03T21:17:10.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
38592025-10-03T21:17:10.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38602025-10-03T21:17:10.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38612025-10-03T21:17:10.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38622025-10-03T21:17:10.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38632025-10-03T21:17:10.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38642025-10-03T21:17:10.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
38652025-10-03T21:17:10.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38662025-10-03T21:17:10.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38672025-10-03T21:17:10.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38682025-10-03T21:17:10.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38692025-10-03T21:17:10.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38702025-10-03T21:17:10.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38712025-10-03T21:17:10.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38722025-10-03T21:17:10.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38732025-10-03T21:17:10.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38742025-10-03T21:17:10.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38752025-10-03T21:17:10.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38762025-10-03T21:17:10.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38772025-10-03T21:17:10.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38782025-10-03T21:17:10.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
38792025-10-03T21:17:10.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38802025-10-03T21:17:10.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38812025-10-03T21:17:10.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38822025-10-03T21:17:10.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38832025-10-03T21:17:10.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38842025-10-03T21:17:10.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38852025-10-03T21:17:10.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38862025-10-03T21:17:10.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38872025-10-03T21:17:10.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
38882025-10-03T21:17:10.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38892025-10-03T21:17:10.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38902025-10-03T21:17:10.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
38912025-10-03T21:17:10.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38922025-10-03T21:17:10.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38932025-10-03T21:17:10.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38942025-10-03T21:17:10.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38952025-10-03T21:17:10.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38962025-10-03T21:17:10.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38972025-10-03T21:17:10.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38982025-10-03T21:17:10.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38992025-10-03T21:17:10.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39002025-10-03T21:17:10.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39012025-10-03T21:17:10.782ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
39022025-10-03T21:17:10.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39032025-10-03T21:17:10.782ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "a4bc84fd-2621-4e0b-943b-463770a10dba", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
39042025-10-03T21:17:10.782ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
39052025-10-03T21:17:10.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
39062025-10-03T21:17:10.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39072025-10-03T21:17:10.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39082025-10-03T21:17:10.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39092025-10-03T21:17:10.784ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "916f05bb-693b-4155-b37d-97827dfe7ded", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
39102025-10-03T21:17:10.784ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
39112025-10-03T21:17:10.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39122025-10-03T21:17:10.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39132025-10-03T21:17:10.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39142025-10-03T21:17:10.789ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
39152025-10-03T21:17:10.789ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 598.348524ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
39162025-10-03T21:17:10.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39172025-10-03T21:17:10.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39182025-10-03T21:17:10.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39192025-10-03T21:17:10.799ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
39202025-10-03T21:17:10.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39212025-10-03T21:17:10.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39222025-10-03T21:17:10.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39232025-10-03T21:17:10.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39242025-10-03T21:17:10.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39252025-10-03T21:17:10.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39262025-10-03T21:17:10.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39272025-10-03T21:17:10.808ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
39282025-10-03T21:17:10.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39292025-10-03T21:17:10.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39302025-10-03T21:17:10.816ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(bae9c0a0-bcad-48fe-9870-38aaaaabc637, 0), (aae16669-bf15-4c64-abd0-29f252338280, 0), (128c8459-d400-4e28-81e8-71ae30f5474f, 0)]
39312025-10-03T21:17:10.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39322025-10-03T21:17:10.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39332025-10-03T21:17:10.817ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5076, tv_nsec: 305714729 } })
39342025-10-03T21:17:10.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39352025-10-03T21:17:10.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39362025-10-03T21:17:10.820ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
39372025-10-03T21:17:10.820ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
39382025-10-03T21:17:10.820ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = ca616112-757a-4b43-81aa-87c32bc778b0 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
39392025-10-03T21:17:10.820ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
39402025-10-03T21:17:10.820ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 629.302864ms iteration = 24 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
39412025-10-03T21:17:10.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39422025-10-03T21:17:10.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39432025-10-03T21:17:10.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39442025-10-03T21:17:10.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39452025-10-03T21:17:10.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39462025-10-03T21:17:10.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39472025-10-03T21:17:10.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39482025-10-03T21:17:10.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39492025-10-03T21:17:10.836ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
39502025-10-03T21:17:10.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39512025-10-03T21:17:10.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39522025-10-03T21:17:10.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
39532025-10-03T21:17:10.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39542025-10-03T21:17:10.838ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
39552025-10-03T21:17:10.839ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
39562025-10-03T21:17:10.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
39572025-10-03T21:17:10.840ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
39582025-10-03T21:17:10.841ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae087bd2-eb63-4b2f-9301-5bae1a93297d", "content-length": "691", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
39592025-10-03T21:17:10.841ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39602025-10-03T21:17:10.841ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
39612025-10-03T21:17:10.841ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
39622025-10-03T21:17:10.841ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
39632025-10-03T21:17:10.843ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 652.472625ms iteration = 21 status = {}
39642025-10-03T21:17:10.843ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
39652025-10-03T21:17:10.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39662025-10-03T21:17:10.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39672025-10-03T21:17:10.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
39682025-10-03T21:17:10.844ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
39692025-10-03T21:17:10.845ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "671f7039-7891-4c49-9992-aad3fd30b651", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
39702025-10-03T21:17:10.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39712025-10-03T21:17:10.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39722025-10-03T21:17:10.847ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
39732025-10-03T21:17:10.847ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 655.63687ms
39742025-10-03T21:17:10.847ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 655.652121ms iteration = 3 status = {}
39752025-10-03T21:17:10.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39762025-10-03T21:17:10.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39772025-10-03T21:17:10.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39782025-10-03T21:17:10.853ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
39792025-10-03T21:17:10.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39802025-10-03T21:17:10.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39812025-10-03T21:17:10.860ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
39822025-10-03T21:17:10.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39832025-10-03T21:17:10.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39842025-10-03T21:17:10.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39852025-10-03T21:17:10.864ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
39862025-10-03T21:17:10.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39872025-10-03T21:17:10.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39882025-10-03T21:17:10.866ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
39892025-10-03T21:17:10.866ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
39902025-10-03T21:17:10.866ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
39912025-10-03T21:17:10.939ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
39922025-10-03T21:17:10.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
39932025-10-03T21:17:10.940ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), time_created: 2025-10-03T21:07:05.994Z, time_modified: 2025-10-03T21:07:05.994Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
39942025-10-03T21:17:10.940ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
39952025-10-03T21:17:10.942ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 751.23555ms iteration = 21 status = null
39962025-10-03T21:17:10.953ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
39972025-10-03T21:17:10.953ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
39982025-10-03T21:17:10.954ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb011f68-c5a7-488f-a015-cdf977a64dad", "content-length": "372", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
39992025-10-03T21:17:10.954ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40002025-10-03T21:17:10.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40012025-10-03T21:17:10.970ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("26cdd4d2-d0ac-43df-bf3e-350fd160efec.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41cf4311-4781-41d3-b8bc-181c344891b0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b55c423c-6cde-4185-879d-3a1c4cf01de4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b807e991-1a36-4877-8937-3efb3994dbc2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::3)) }], valid_until: Instant { tv_sec: 5076, tv_nsec: 459411624 } })
40022025-10-03T21:17:10.971ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
40032025-10-03T21:17:10.971ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
40042025-10-03T21:17:10.971ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
40052025-10-03T21:17:10.972ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "eeb0eb73-f065-42d1-95a0-173c535c6c51", "content-length": "476", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40062025-10-03T21:17:10.972ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
40072025-10-03T21:17:10.972ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c46d093e-8d98-483b-a4cb-e1904a8fffe1", "content-length": "734", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40082025-10-03T21:17:10.973ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
40092025-10-03T21:17:10.973ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1cd6f34-425e-4f53-aa83-480cdeccc649", "content-length": "111", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40102025-10-03T21:17:10.973ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
40112025-10-03T21:17:10.973ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e6510e9-170e-41a2-8d13-90869c058e5a", "content-length": "111", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40122025-10-03T21:17:10.973ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
40132025-10-03T21:17:10.974ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd9e338e-6c13-46cd-bf39-4bfb988eb309", "content-length": "163", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40142025-10-03T21:17:10.974ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
40152025-10-03T21:17:10.974ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a13dc38-18cb-45f5-b379-7dc7fa9f751c", "content-length": "163", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40162025-10-03T21:17:10.974ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
40172025-10-03T21:17:10.975ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03a2bc49-ad56-4ed0-9fbb-4dc54aefbd41", "content-length": "166", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40182025-10-03T21:17:10.975ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
40192025-10-03T21:17:10.975ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed307eec-7094-4ee2-8ecc-08bf5b2370a3", "content-length": "166", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40202025-10-03T21:17:10.975ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
40212025-10-03T21:17:10.975ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "493be42d-d042-49fd-9406-2f787863dab8", "content-length": "702", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40222025-10-03T21:17:10.975ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
40232025-10-03T21:17:10.975ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f1ce769-534a-4709-b42e-ac8f87c54feb", "content-length": "727", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40242025-10-03T21:17:10.975ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
40252025-10-03T21:17:10.976ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ec2745a-0ff0-4fcd-a3f8-401398fb7afb", "content-length": "729", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40262025-10-03T21:17:10.976ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
40272025-10-03T21:17:10.976ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "778d2dbc-9667-44bb-9546-4a5e2c755f96", "content-length": "728", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40282025-10-03T21:17:10.976ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
40292025-10-03T21:17:10.976ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a92e578a-cc71-486b-950c-15d931c67e34", "content-length": "722", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40302025-10-03T21:17:10.976ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
40312025-10-03T21:17:10.977ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "87980875-503b-462d-ad06-4666705f1df6", "content-length": "10", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40322025-10-03T21:17:10.977ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
40332025-10-03T21:17:10.977ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eca021a0-0cca-4b77-9e6d-e36fcc347490", "content-length": "144", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40342025-10-03T21:17:10.977ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
40352025-10-03T21:17:10.977ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c404a7a4-e1e5-4a56-a602-c801c7085d1a", "content-length": "147", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40362025-10-03T21:17:10.977ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
40372025-10-03T21:17:10.978ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f23bf4ec-5559-4f3b-8610-8b60f542856d", "content-length": "109", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40382025-10-03T21:17:10.978ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
40392025-10-03T21:17:10.978ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0df3589-4c16-4883-abe4-1805e15509b4", "content-length": "109", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40402025-10-03T21:17:10.978ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
40412025-10-03T21:17:10.979ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35507952-5034-4a13-b0ad-9c48adeddc3c", "content-length": "163", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40422025-10-03T21:17:10.979ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
40432025-10-03T21:17:10.979ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44aea97f-6967-42da-9212-2abecc8fac72", "content-length": "163", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40442025-10-03T21:17:10.979ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
40452025-10-03T21:17:10.980ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "028b423a-d722-4179-9400-d4ad33322791", "content-length": "166", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40462025-10-03T21:17:10.980ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
40472025-10-03T21:17:10.980ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff54aa74-7544-46c6-82b9-83799cced3f2", "content-length": "166", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40482025-10-03T21:17:10.980ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
40492025-10-03T21:17:10.980ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1226df50-b45f-4318-8d28-d1890ef33866", "content-length": "702", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40502025-10-03T21:17:10.980ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
40512025-10-03T21:17:10.981ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5d5d798-fd19-4df7-9bc1-ae13275fcb32", "content-length": "727", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40522025-10-03T21:17:10.981ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
40532025-10-03T21:17:10.981ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92ddf9c3-7960-4b4a-8957-3b330d484129", "content-length": "729", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40542025-10-03T21:17:10.981ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
40552025-10-03T21:17:10.981ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1acddd8-8128-45a3-b08d-7c82d0e9cf85", "content-length": "728", "date": "Fri, 03 Oct 2025 21:17:10 GMT"} })
40562025-10-03T21:17:10.981ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40572025-10-03T21:17:10.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
40582025-10-03T21:17:10.983ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40592025-10-03T21:17:10.986ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
40602025-10-03T21:17:10.986ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40612025-10-03T21:17:10.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
40622025-10-03T21:17:10.989ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40632025-10-03T21:17:10.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
40642025-10-03T21:17:10.991ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40652025-10-03T21:17:10.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
40662025-10-03T21:17:11.016ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
40672025-10-03T21:17:11.016ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
40682025-10-03T21:17:11.018ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ac9c2a34-f97b-48a6-9362-e1eba28549e3", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
40692025-10-03T21:17:11.018ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40702025-10-03T21:17:11.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40712025-10-03T21:17:11.022ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40722025-10-03T21:17:11.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40732025-10-03T21:17:11.025ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40742025-10-03T21:17:11.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40752025-10-03T21:17:11.032ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
40762025-10-03T21:17:11.032ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40772025-10-03T21:17:11.037ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "25627d4b-58a6-466a-8c39-08d475c568b5", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
40782025-10-03T21:17:11.040ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 849.150066ms iteration = 21 status = {}
40792025-10-03T21:17:11.044ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40802025-10-03T21:17:11.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40812025-10-03T21:17:11.046ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
40822025-10-03T21:17:11.046ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "0aff4e3b-9c6e-4842-9a40-49d42fc2f3bb", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40832025-10-03T21:17:11.046ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40842025-10-03T21:17:11.047ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
40852025-10-03T21:17:11.047ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40862025-10-03T21:17:11.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40872025-10-03T21:17:11.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40882025-10-03T21:17:11.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40892025-10-03T21:17:11.049ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "159358ba-b70b-409e-baf5-87072f6b412b", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40902025-10-03T21:17:11.049ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40912025-10-03T21:17:11.050ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40922025-10-03T21:17:11.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40932025-10-03T21:17:11.051ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "169f7b34-05da-488d-932c-8f3ba895578b", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40942025-10-03T21:17:11.051ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40952025-10-03T21:17:11.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40962025-10-03T21:17:11.053ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40972025-10-03T21:17:11.053ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 862.377545ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
40982025-10-03T21:17:11.056ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
40992025-10-03T21:17:11.056ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
41002025-10-03T21:17:11.062ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "b90b9795-6d4c-4030-a004-f477f228d3eb", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
41012025-10-03T21:17:11.064ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
41022025-10-03T21:17:11.064ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
41032025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
41042025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
41052025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41062025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41072025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41082025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41092025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
41102025-10-03T21:17:11.065ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = Dns step id = Ensure
41112025-10-03T21:17:11.065ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
41122025-10-03T21:17:11.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41132025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41142025-10-03T21:17:11.065ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41152025-10-03T21:17:11.066ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
41162025-10-03T21:17:11.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41172025-10-03T21:17:11.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41182025-10-03T21:17:11.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41192025-10-03T21:17:11.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41202025-10-03T21:17:11.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41212025-10-03T21:17:11.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41222025-10-03T21:17:11.069ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
41232025-10-03T21:17:11.069ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
41242025-10-03T21:17:11.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41252025-10-03T21:17:11.072ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
41262025-10-03T21:17:11.072ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
41272025-10-03T21:17:11.072ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
41282025-10-03T21:17:11.072ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
41292025-10-03T21:17:11.072ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
41302025-10-03T21:17:11.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41312025-10-03T21:17:11.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41322025-10-03T21:17:11.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41332025-10-03T21:17:11.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41342025-10-03T21:17:11.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41352025-10-03T21:17:11.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41362025-10-03T21:17:11.086ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
41372025-10-03T21:17:11.086ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
41382025-10-03T21:17:11.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41392025-10-03T21:17:11.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41402025-10-03T21:17:11.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41412025-10-03T21:17:11.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41422025-10-03T21:17:11.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41432025-10-03T21:17:11.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41442025-10-03T21:17:11.097ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
41452025-10-03T21:17:11.097ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
41462025-10-03T21:17:11.097ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
41472025-10-03T21:17:11.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41482025-10-03T21:17:11.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41492025-10-03T21:17:11.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41502025-10-03T21:17:11.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41512025-10-03T21:17:11.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41522025-10-03T21:17:11.103ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41532025-10-03T21:17:11.104ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
41542025-10-03T21:17:11.104ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
41552025-10-03T21:17:11.104ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
41562025-10-03T21:17:11.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41572025-10-03T21:17:11.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41582025-10-03T21:17:11.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41592025-10-03T21:17:11.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41602025-10-03T21:17:11.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41612025-10-03T21:17:11.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41622025-10-03T21:17:11.112ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
41632025-10-03T21:17:11.112ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
41642025-10-03T21:17:11.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41652025-10-03T21:17:11.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41662025-10-03T21:17:11.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41672025-10-03T21:17:11.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41682025-10-03T21:17:11.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41692025-10-03T21:17:11.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41702025-10-03T21:17:11.120ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
41712025-10-03T21:17:11.121ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
41722025-10-03T21:17:11.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41732025-10-03T21:17:11.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41742025-10-03T21:17:11.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41752025-10-03T21:17:11.124ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
41762025-10-03T21:17:11.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41772025-10-03T21:17:11.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41782025-10-03T21:17:11.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41792025-10-03T21:17:11.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41802025-10-03T21:17:11.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41812025-10-03T21:17:11.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41822025-10-03T21:17:11.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41832025-10-03T21:17:11.139ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41842025-10-03T21:17:11.141ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
41852025-10-03T21:17:11.141ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
41862025-10-03T21:17:11.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = OmicronZones step id = Cleanup
41872025-10-03T21:17:11.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = OmicronZones step id = Cleanup
41882025-10-03T21:17:11.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = PhysicalDisks step id = Cleanup
41892025-10-03T21:17:11.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = Clickhouse step id = Ensure
41902025-10-03T21:17:11.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = Clickhouse step id = Ensure
41912025-10-03T21:17:11.155ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb44f43b-784f-4714-9e5f-30dd19a11fe7", "content-length": "88632", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
41922025-10-03T21:17:11.157ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
41932025-10-03T21:17:11.157ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
41942025-10-03T21:17:11.157ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
41952025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
41962025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
41972025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
41982025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
41992025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
42002025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b09b6cc-7372-43f9-9487-bdb9984289b1", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42012025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e9cf53f-934b-412c-909b-41e720288559", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42022025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
42032025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8491d92-489f-49d5-a375-87e0ee4fff0a", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42042025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
42052025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1aebc666-9143-44f7-be03-ac8d593f9dcb", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42062025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
42072025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
42082025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "868d74d9-bca3-4f90-99d8-326dde59de5c", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42092025-10-03T21:17:11.157ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
42102025-10-03T21:17:11.181ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
42112025-10-03T21:17:11.186ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "353ba0a1-3a5b-4489-b669-c1c825e6682e", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42122025-10-03T21:17:11.188ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = SupportBundles step id = Cleanup
42132025-10-03T21:17:11.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42142025-10-03T21:17:11.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42152025-10-03T21:17:11.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42162025-10-03T21:17:11.192ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
42172025-10-03T21:17:11.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42182025-10-03T21:17:11.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42192025-10-03T21:17:11.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42202025-10-03T21:17:11.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42212025-10-03T21:17:11.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42222025-10-03T21:17:11.198ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd49bd37-fade-49e2-847e-39060ddcecb2", "content-length": "396729", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42232025-10-03T21:17:11.202ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "704a78e8-011e-4296-8e6d-db5c9f1c23c5", "content-length": "403915", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42242025-10-03T21:17:11.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
42252025-10-03T21:17:11.204ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d19a0992-5e58-4c32-bf95-bfdc9a25d073", "content-length": "404549", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42262025-10-03T21:17:11.204ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "854c3d71-6ef4-4729-8118-054dd084a652", "content-length": "407254", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42272025-10-03T21:17:11.205ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "62c305fa-8ac6-4304-b510-93f371df039a", "content-length": "431274", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42282025-10-03T21:17:11.207ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = OmicronZones step id = Cleanup
42292025-10-03T21:17:11.207ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
42302025-10-03T21:17:11.207ZDEBGnexus (BackgroundTasks): re-assign sagas: found Nexus instances background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
42312025-10-03T21:17:11.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42322025-10-03T21:17:11.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42332025-10-03T21:17:11.209ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
42342025-10-03T21:17:11.209ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
42352025-10-03T21:17:11.209ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
42362025-10-03T21:17:11.209ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
42372025-10-03T21:17:11.209ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
42382025-10-03T21:17:11.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42392025-10-03T21:17:11.212ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
42402025-10-03T21:17:11.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42412025-10-03T21:17:11.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42422025-10-03T21:17:11.224ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 491 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = e68de248-9ca7-439f-bab1-6b6223c53282 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
42432025-10-03T21:17:11.246ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 (service)
42442025-10-03T21:17:11.246ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
42452025-10-03T21:17:11.250ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d18b08c-e109-4b89-99d5-242b142b4a0e", "content-length": "123", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42462025-10-03T21:17:11.252ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
42472025-10-03T21:17:11.254ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:54 nexus_zone_ids = []
42482025-10-03T21:17:11.255ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(675bea75-8e2d-4986-86f2-b590a30bb084 (blueprint), false)
42492025-10-03T21:17:11.255ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = Cockroach step id = Ensure
42502025-10-03T21:17:11.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42512025-10-03T21:17:11.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42522025-10-03T21:17:11.270ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
42532025-10-03T21:17:11.270ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1da42034-8d3d-4b9b-8416-537d30068e62 step component = MgsUpdates step id = Ensure
42542025-10-03T21:17:11.270ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 32 reason = Dependency
42552025-10-03T21:17:11.270ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42562025-10-03T21:17:11.270ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.079953141s iteration = 12 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":5070,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":177461364,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":177461284,"secs":0}},"event_index":1,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":177517644,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47967731,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":47967691,"secs":0}},"event_index":2,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":225504325,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":57802453,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":57802403,"secs":0}},"event_index":3,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":283383639,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":253736664,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":253736554,"secs":0}},"event_index":4,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":537141493,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":337542240,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":337542190,"secs":0}},"event_index":5,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":874700653,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":75796965,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":75796915,"secs":0}},"event_index":6,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":950519889,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4531,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4491,"secs":0}},"event_index":7,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":950542330,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2731,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2691,"secs":0}},"event_index":8,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":950547679,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3611,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3581,"secs":0}},"event_index":9,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":950552700,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3020,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2380,"secs":0}},"event_index":10,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":950556990,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":46582765,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":46582725,"secs":0}},"event_index":11,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":997141825,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":19324218,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":19324178,"secs":0}},"event_index":12,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":16476754,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47548687,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":47548647,"secs":0}},"event_index":13,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":64041341,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":15470894,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":15470854,"secs":0}},"event_index":14,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":79528646,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4450,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4420,"secs":0}},"event_index":15,"execution_id":"ce285881-1d84-41ba-a878-18423f2b70b5","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":79541116,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"675bea75-8e2d-4986-86f2-b590a30bb084"}
42572025-10-03T21:17:11.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42582025-10-03T21:17:11.272ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42592025-10-03T21:17:11.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
42602025-10-03T21:17:11.274ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42612025-10-03T21:17:11.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42622025-10-03T21:17:11.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42632025-10-03T21:17:11.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42642025-10-03T21:17:11.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42652025-10-03T21:17:11.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
42662025-10-03T21:17:11.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42672025-10-03T21:17:11.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42682025-10-03T21:17:11.284ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
42692025-10-03T21:17:11.284ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 13.529623ms iteration = 32 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
42702025-10-03T21:17:11.307ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9f9149f0-794c-46e9-bf18-23ee0d1ec178", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42712025-10-03T21:17:11.307ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
42722025-10-03T21:17:11.307ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
42732025-10-03T21:17:11.314ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "87310ddf-43a6-47a3-a03d-f80c22137080", "content-length": "682", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42742025-10-03T21:17:11.314ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42752025-10-03T21:17:11.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42762025-10-03T21:17:11.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42772025-10-03T21:17:11.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42782025-10-03T21:17:11.322ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42792025-10-03T21:17:11.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42802025-10-03T21:17:11.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42812025-10-03T21:17:11.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42822025-10-03T21:17:11.330ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.139020868s iteration = 21 status = {}
42832025-10-03T21:17:11.348ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
42842025-10-03T21:17:11.348ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
42852025-10-03T21:17:11.349ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc8a0c88-130c-4a69-b15c-888041417b52", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42862025-10-03T21:17:11.349ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
42872025-10-03T21:17:11.351ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
42882025-10-03T21:17:11.351ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
42892025-10-03T21:17:11.351ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "89d6c32b-1e34-4be5-ad84-abb673b75b88", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42902025-10-03T21:17:11.351ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
42912025-10-03T21:17:11.353ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
42922025-10-03T21:17:11.353ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
42932025-10-03T21:17:11.355ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc5006dd-f85b-4735-86aa-f9397e3acc33", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:17:11 GMT"} })
42942025-10-03T21:17:11.355ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
42952025-10-03T21:17:11.357ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
42962025-10-03T21:17:11.357ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
42972025-10-03T21:17:11.359ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42982025-10-03T21:17:11.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
42992025-10-03T21:17:11.362ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43002025-10-03T21:17:11.364ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
43012025-10-03T21:17:11.364ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43022025-10-03T21:17:11.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43032025-10-03T21:17:11.560ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = a4eb63c6-df9b-4156-906c-fe9d51e6323f file = nexus/db-queries/src/db/datastore/inventory.rs:1614
43042025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = a4eb63c6-df9b-4156-906c-fe9d51e6323f time_started = 2025-10-03 21:17:10.971888 UTC
43052025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 25 reason = Dependency
43062025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Dependency
43072025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43082025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 15 reason = Dependency
43092025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43102025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43112025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.368308426s iteration = 12 status = {"collection_id":"a4eb63c6-df9b-4156-906c-fe9d51e6323f","time_done":"2025-10-03 21:17:11.357684 UTC","time_started":"2025-10-03 21:17:10.971888 UTC"}
43122025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Dependency
43132025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43142025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43152025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43162025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43172025-10-03T21:17:11.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43182025-10-03T21:17:11.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43192025-10-03T21:17:11.562ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43202025-10-03T21:17:11.562ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43212025-10-03T21:17:11.565ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
43222025-10-03T21:17:11.565ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43232025-10-03T21:17:11.565ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
43242025-10-03T21:17:11.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43252025-10-03T21:17:11.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43262025-10-03T21:17:11.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43272025-10-03T21:17:11.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43282025-10-03T21:17:11.566ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43292025-10-03T21:17:11.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43302025-10-03T21:17:11.567ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43312025-10-03T21:17:11.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43322025-10-03T21:17:11.569ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43332025-10-03T21:17:11.571ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43342025-10-03T21:17:11.571ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43352025-10-03T21:17:11.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43362025-10-03T21:17:11.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43372025-10-03T21:17:11.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43382025-10-03T21:17:11.575ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(bae9c0a0-bcad-48fe-9870-38aaaaabc637, 0), (aae16669-bf15-4c64-abd0-29f252338280, 0), (128c8459-d400-4e28-81e8-71ae30f5474f, 0), (a4eb63c6-df9b-4156-906c-fe9d51e6323f, 0)] collection_id = bae9c0a0-bcad-48fe-9870-38aaaaabc637
43392025-10-03T21:17:11.575ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43402025-10-03T21:17:11.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43412025-10-03T21:17:11.590ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 30.743757ms iteration = 33 status = {"physical_disks_added":0}
43422025-10-03T21:17:11.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43432025-10-03T21:17:11.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43442025-10-03T21:17:11.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43452025-10-03T21:17:11.623ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
43462025-10-03T21:17:11.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43472025-10-03T21:17:11.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43482025-10-03T21:17:11.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43492025-10-03T21:17:11.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43502025-10-03T21:17:11.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43512025-10-03T21:17:11.627ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
43522025-10-03T21:17:11.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43532025-10-03T21:17:11.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43542025-10-03T21:17:11.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43552025-10-03T21:17:11.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43562025-10-03T21:17:11.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43572025-10-03T21:17:11.629ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
43582025-10-03T21:17:11.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43592025-10-03T21:17:11.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43602025-10-03T21:17:11.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43612025-10-03T21:17:11.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43622025-10-03T21:17:11.633ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
43632025-10-03T21:17:11.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43642025-10-03T21:17:11.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43652025-10-03T21:17:11.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
43662025-10-03T21:17:11.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43672025-10-03T21:17:11.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43682025-10-03T21:17:11.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43692025-10-03T21:17:11.636ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
43702025-10-03T21:17:11.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43712025-10-03T21:17:11.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43722025-10-03T21:17:11.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43732025-10-03T21:17:11.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43742025-10-03T21:17:11.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43752025-10-03T21:17:11.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43762025-10-03T21:17:11.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
43772025-10-03T21:17:11.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43782025-10-03T21:17:11.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43792025-10-03T21:17:11.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43802025-10-03T21:17:11.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43812025-10-03T21:17:11.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43822025-10-03T21:17:11.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43832025-10-03T21:17:11.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
43842025-10-03T21:17:11.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
43852025-10-03T21:17:11.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
43862025-10-03T21:17:11.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43872025-10-03T21:17:11.646ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
43882025-10-03T21:17:11.646ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 86.233174ms iteration = 15 status = {"blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","inventory_collection_id":"31053175-4bf3-48bb-a958-177800eb5a29","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
43892025-10-03T21:17:11.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
43902025-10-03T21:17:11.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43912025-10-03T21:17:11.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43922025-10-03T21:17:11.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43932025-10-03T21:17:11.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43942025-10-03T21:17:11.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43952025-10-03T21:17:11.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
43962025-10-03T21:17:11.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43972025-10-03T21:17:11.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43982025-10-03T21:17:11.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43992025-10-03T21:17:11.657ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44002025-10-03T21:17:11.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44012025-10-03T21:17:11.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44022025-10-03T21:17:11.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44032025-10-03T21:17:11.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44042025-10-03T21:17:11.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44052025-10-03T21:17:11.664ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
44062025-10-03T21:17:11.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44072025-10-03T21:17:11.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44082025-10-03T21:17:11.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44092025-10-03T21:17:11.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44102025-10-03T21:17:11.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44112025-10-03T21:17:11.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44122025-10-03T21:17:11.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44132025-10-03T21:17:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44142025-10-03T21:17:11.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44152025-10-03T21:17:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44162025-10-03T21:17:11.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44172025-10-03T21:17:11.677ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44182025-10-03T21:17:11.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44192025-10-03T21:17:11.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44202025-10-03T21:17:11.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44212025-10-03T21:17:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44222025-10-03T21:17:11.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44232025-10-03T21:17:11.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44242025-10-03T21:17:11.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44252025-10-03T21:17:11.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44262025-10-03T21:17:11.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44272025-10-03T21:17:11.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44282025-10-03T21:17:11.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44292025-10-03T21:17:11.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44302025-10-03T21:17:11.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44312025-10-03T21:17:11.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44322025-10-03T21:17:11.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44332025-10-03T21:17:11.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44342025-10-03T21:17:11.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44352025-10-03T21:17:11.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44362025-10-03T21:17:11.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44372025-10-03T21:17:11.705ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
44382025-10-03T21:17:11.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44392025-10-03T21:17:11.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44402025-10-03T21:17:11.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44412025-10-03T21:17:11.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44422025-10-03T21:17:11.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44432025-10-03T21:17:11.713ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
44442025-10-03T21:17:11.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44452025-10-03T21:17:11.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44462025-10-03T21:17:11.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44472025-10-03T21:17:11.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44482025-10-03T21:17:11.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44492025-10-03T21:17:11.721ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
44502025-10-03T21:17:11.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44512025-10-03T21:17:11.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44522025-10-03T21:17:11.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44532025-10-03T21:17:11.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44542025-10-03T21:17:11.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44552025-10-03T21:17:11.724ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = bae9c0a0-bcad-48fe-9870-38aaaaabc637 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
44562025-10-03T21:17:11.724ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44572025-10-03T21:17:11.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
44582025-10-03T21:17:11.727ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
44592025-10-03T21:17:11.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44602025-10-03T21:17:11.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44612025-10-03T21:17:11.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44622025-10-03T21:17:11.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44632025-10-03T21:17:11.730ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(aae16669-bf15-4c64-abd0-29f252338280, 0), (128c8459-d400-4e28-81e8-71ae30f5474f, 0), (a4eb63c6-df9b-4156-906c-fe9d51e6323f, 0), (eadaeff6-9ae8-4553-bd1f-b9b3d921e683, 0)] collection_id = aae16669-bf15-4c64-abd0-29f252338280
44642025-10-03T21:17:11.730ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44652025-10-03T21:17:11.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
44662025-10-03T21:17:11.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44672025-10-03T21:17:11.733ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
44682025-10-03T21:17:11.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44692025-10-03T21:17:11.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44702025-10-03T21:17:11.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44712025-10-03T21:17:11.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44722025-10-03T21:17:11.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44732025-10-03T21:17:11.739ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
44742025-10-03T21:17:11.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44752025-10-03T21:17:11.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44762025-10-03T21:17:11.774ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
44772025-10-03T21:17:11.774ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
44782025-10-03T21:17:11.774ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 557c8c6c-554f-470b-a036-b8e1cd0c11b5 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
44792025-10-03T21:17:11.774ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
44802025-10-03T21:17:11.774ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 214.254338ms iteration = 25 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
44812025-10-03T21:17:11.860ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 45 transaction = inventory_delete_collection
44822025-10-03T21:17:11.951ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
44832025-10-03T21:17:11.951ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = aae16669-bf15-4c64-abd0-29f252338280 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
44842025-10-03T21:17:11.951ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
44852025-10-03T21:17:11.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
44862025-10-03T21:17:11.987ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(a4eb63c6-df9b-4156-906c-fe9d51e6323f, 0), (eadaeff6-9ae8-4553-bd1f-b9b3d921e683, 0), (31053175-4bf3-48bb-a958-177800eb5a29, 0)]
44872025-10-03T21:17:11.987ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5077, tv_nsec: 476178001 } })
44882025-10-03T21:17:12.030ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("26cdd4d2-d0ac-43df-bf3e-350fd160efec.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41cf4311-4781-41d3-b8bc-181c344891b0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b55c423c-6cde-4185-879d-3a1c4cf01de4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b807e991-1a36-4877-8937-3efb3994dbc2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::3)) }], valid_until: Instant { tv_sec: 5077, tv_nsec: 518466929 } })
44892025-10-03T21:17:12.030ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
44902025-10-03T21:17:12.030ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
44912025-10-03T21:17:12.030ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
44922025-10-03T21:17:12.031ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c71cbb6-1a8f-4d0f-ab2b-831f5652af59", "content-length": "476", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
44932025-10-03T21:17:12.031ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
44942025-10-03T21:17:12.031ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6df6c02c-3304-4c9f-b7da-93d0f68a9ead", "content-length": "734", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
44952025-10-03T21:17:12.031ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
44962025-10-03T21:17:12.032ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "76689890-4372-4d5c-a059-2c5342936412", "content-length": "111", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
44972025-10-03T21:17:12.032ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
44982025-10-03T21:17:12.032ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a29d096-0040-42a2-b3db-731e7c89b80e", "content-length": "111", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
44992025-10-03T21:17:12.032ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
45002025-10-03T21:17:12.033ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "219c8dea-ff64-4efe-900d-d6976d2c740b", "content-length": "163", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45012025-10-03T21:17:12.033ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
45022025-10-03T21:17:12.033ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d4f0de1-0f1f-4250-8bf0-400d358c5475", "content-length": "163", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45032025-10-03T21:17:12.033ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
45042025-10-03T21:17:12.033ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d61be16-709f-453e-b80a-3c2b7751c28b", "content-length": "166", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45052025-10-03T21:17:12.033ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
45062025-10-03T21:17:12.034ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f0adfe7-4866-4552-96d1-9303175bca58", "content-length": "166", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45072025-10-03T21:17:12.034ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
45082025-10-03T21:17:12.034ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a705a0d-2ad5-4752-a9bd-b0c48187a78b", "content-length": "702", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45092025-10-03T21:17:12.034ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
45102025-10-03T21:17:12.034ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "67cd89f7-059d-4ca4-bc18-d356ea16521e", "content-length": "727", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45112025-10-03T21:17:12.034ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
45122025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4026025c-1560-4d5a-8bfc-28f22534e512", "content-length": "729", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45132025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
45142025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fe02866-bfbd-4eaf-9b00-69a74320b53a", "content-length": "728", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45152025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
45162025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b96a838-47f8-427e-ba07-4bac75da84da", "content-length": "722", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45172025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
45182025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbf7a06c-2b57-4e6e-abe8-55ca915a99e0", "content-length": "10", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45192025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
45202025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fafb98db-5e3f-4e1f-9dae-c335871dcccc", "content-length": "144", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45212025-10-03T21:17:12.035ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
45222025-10-03T21:17:12.036ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d66cb90-ce1d-4b8d-a710-68d97f4569ae", "content-length": "147", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45232025-10-03T21:17:12.036ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
45242025-10-03T21:17:12.036ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f55ce392-8cf6-48f4-a07d-04bc0428391c", "content-length": "109", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45252025-10-03T21:17:12.036ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
45262025-10-03T21:17:12.036ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "494e8024-8ac0-47e6-aa5b-84e709c8c6ff", "content-length": "109", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45272025-10-03T21:17:12.036ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
45282025-10-03T21:17:12.037ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa5f1e64-6958-459a-be6f-001f121ead3d", "content-length": "163", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45292025-10-03T21:17:12.037ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
45302025-10-03T21:17:12.037ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "56ad4dd2-4d9e-495d-befc-fbabc025ba36", "content-length": "163", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45312025-10-03T21:17:12.037ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
45322025-10-03T21:17:12.037ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3256e601-fdd4-44b6-8a25-06deb3cb6451", "content-length": "166", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45332025-10-03T21:17:12.037ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
45342025-10-03T21:17:12.038ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fa00d8f-5212-47d0-9349-5c40512e2799", "content-length": "166", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45352025-10-03T21:17:12.038ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
45362025-10-03T21:17:12.038ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88fad635-6bca-4bf5-9166-53cd731b8c67", "content-length": "702", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45372025-10-03T21:17:12.038ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
45382025-10-03T21:17:12.038ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47423dcd-9da8-4b14-abba-f2d56cd85a25", "content-length": "727", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45392025-10-03T21:17:12.038ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
45402025-10-03T21:17:12.038ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a55a18c-f21f-47de-8d6a-9c4bc67b9f74", "content-length": "729", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45412025-10-03T21:17:12.038ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
45422025-10-03T21:17:12.039ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b1a9a56-9d1f-4b67-8ff8-34b9f327e791", "content-length": "728", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45432025-10-03T21:17:12.039ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45442025-10-03T21:17:12.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
45452025-10-03T21:17:12.040ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45462025-10-03T21:17:12.042ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
45472025-10-03T21:17:12.042ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45482025-10-03T21:17:12.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
45492025-10-03T21:17:12.044ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45502025-10-03T21:17:12.046ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
45512025-10-03T21:17:12.046ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45522025-10-03T21:17:12.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
45532025-10-03T21:17:12.079ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
45542025-10-03T21:17:12.079ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
45552025-10-03T21:17:12.160ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a36674f4-6698-4b5a-b2b1-36b20743724f", "content-length": "88632", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45562025-10-03T21:17:12.162ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
45572025-10-03T21:17:12.162ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
45582025-10-03T21:17:12.162ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
45592025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
45602025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
45612025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
45622025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
45632025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
45642025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "87bcf1b3-42fc-4977-ba33-45269914de09", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45652025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "275530ac-e7a3-4ebe-8ae8-354f9ab4b3cb", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45662025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dead881-03a8-4e39-9178-58c98598d207", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45672025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
45682025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
45692025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
45702025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d83a3fc5-0079-4435-af10-8f23fdb02a53", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45712025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5055af6d-51f6-48fb-8a7e-85994847f2da", "content-length": "64", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45722025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
45732025-10-03T21:17:12.162ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
45742025-10-03T21:17:12.205ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "51a60e82-de4f-4a47-a505-7ad728c9ff2e", "content-length": "404493", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45752025-10-03T21:17:12.205ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2b2dc86-e63a-4550-a5a8-f0c315766020", "content-length": "431407", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45762025-10-03T21:17:12.207ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "02f0f960-01af-4807-a2c7-e8a9f31ba26e", "content-length": "396972", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45772025-10-03T21:17:12.207ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "88c14542-d2e9-4c23-8c33-a83bd5946bdf", "content-length": "407535", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45782025-10-03T21:17:12.210ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6ce71ef-8718-4145-809e-99818296732d", "content-length": "404769", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45792025-10-03T21:17:12.213ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
45802025-10-03T21:17:12.213ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
45812025-10-03T21:17:12.213ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
45822025-10-03T21:17:12.213ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
45832025-10-03T21:17:12.213ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
45842025-10-03T21:17:12.243ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 (service)
45852025-10-03T21:17:12.243ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
45862025-10-03T21:17:12.246ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "43f77ca8-48c9-4281-b4f4-01070963457d", "content-length": "123", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45872025-10-03T21:17:12.248ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
45882025-10-03T21:17:12.340ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
45892025-10-03T21:17:12.340ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
45902025-10-03T21:17:12.341ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa2e17a1-bebf-4051-9915-01f1280bff34", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45912025-10-03T21:17:12.341ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
45922025-10-03T21:17:12.343ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
45932025-10-03T21:17:12.343ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
45942025-10-03T21:17:12.343ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dc165ab-9245-4466-aec0-46a8c27646d2", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45952025-10-03T21:17:12.343ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
45962025-10-03T21:17:12.345ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
45972025-10-03T21:17:12.345ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
45982025-10-03T21:17:12.345ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2fc7b0e-c1c9-4de6-9adf-adb383440c01", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:17:12 GMT"} })
45992025-10-03T21:17:12.346ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
46002025-10-03T21:17:12.348ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
46012025-10-03T21:17:12.348ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
46022025-10-03T21:17:12.350ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
46032025-10-03T21:17:12.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
46042025-10-03T21:17:12.352ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46052025-10-03T21:17:12.354ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
46062025-10-03T21:17:12.354ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
46072025-10-03T21:17:12.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
46082025-10-03T21:17:12.435ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = dffb9ef4-bf37-4870-b52d-2f2d8ae79e87 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
46092025-10-03T21:17:12.435ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = dffb9ef4-bf37-4870-b52d-2f2d8ae79e87 time_started = 2025-10-03 21:17:12.030636 UTC
46102025-10-03T21:17:12.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 26 reason = Dependency
46112025-10-03T21:17:12.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46122025-10-03T21:17:12.435ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Dependency
46132025-10-03T21:17:12.435ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 16 reason = Dependency
46142025-10-03T21:17:12.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
46152025-10-03T21:17:12.435ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46162025-10-03T21:17:12.435ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 875.320011ms iteration = 13 status = {"collection_id":"dffb9ef4-bf37-4870-b52d-2f2d8ae79e87","time_done":"2025-10-03 21:17:12.348167 UTC","time_started":"2025-10-03 21:17:12.030636 UTC"}
46172025-10-03T21:17:12.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
46182025-10-03T21:17:12.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46192025-10-03T21:17:12.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46202025-10-03T21:17:12.438ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46212025-10-03T21:17:12.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46222025-10-03T21:17:12.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46232025-10-03T21:17:12.441ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46242025-10-03T21:17:12.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46252025-10-03T21:17:12.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
46262025-10-03T21:17:12.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
46272025-10-03T21:17:12.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
46282025-10-03T21:17:12.444ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46292025-10-03T21:17:12.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46302025-10-03T21:17:12.445ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46312025-10-03T21:17:12.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46322025-10-03T21:17:12.447ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46332025-10-03T21:17:12.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46342025-10-03T21:17:12.496ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 61.199461ms iteration = 34 status = {"physical_disks_added":0}
46352025-10-03T21:17:12.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46362025-10-03T21:17:12.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46372025-10-03T21:17:12.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46382025-10-03T21:17:12.515ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
46392025-10-03T21:17:12.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46402025-10-03T21:17:12.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46412025-10-03T21:17:12.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46422025-10-03T21:17:12.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
46432025-10-03T21:17:12.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46442025-10-03T21:17:12.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46452025-10-03T21:17:12.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46462025-10-03T21:17:12.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46472025-10-03T21:17:12.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46482025-10-03T21:17:12.592ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
46492025-10-03T21:17:12.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46502025-10-03T21:17:12.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46512025-10-03T21:17:12.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46522025-10-03T21:17:12.596ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
46532025-10-03T21:17:12.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46542025-10-03T21:17:12.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46552025-10-03T21:17:12.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46562025-10-03T21:17:12.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46572025-10-03T21:17:12.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46582025-10-03T21:17:12.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46592025-10-03T21:17:12.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46602025-10-03T21:17:12.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46612025-10-03T21:17:12.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46622025-10-03T21:17:12.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46632025-10-03T21:17:12.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46642025-10-03T21:17:12.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46652025-10-03T21:17:12.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46662025-10-03T21:17:12.613ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
46672025-10-03T21:17:12.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46682025-10-03T21:17:12.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46692025-10-03T21:17:12.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46702025-10-03T21:17:12.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
46712025-10-03T21:17:12.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
46722025-10-03T21:17:12.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46732025-10-03T21:17:12.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46742025-10-03T21:17:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46752025-10-03T21:17:12.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46762025-10-03T21:17:12.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46772025-10-03T21:17:12.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46782025-10-03T21:17:12.624ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
46792025-10-03T21:17:12.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46802025-10-03T21:17:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46812025-10-03T21:17:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46822025-10-03T21:17:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46832025-10-03T21:17:12.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46842025-10-03T21:17:12.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46852025-10-03T21:17:12.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46862025-10-03T21:17:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46872025-10-03T21:17:12.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46882025-10-03T21:17:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46892025-10-03T21:17:12.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46902025-10-03T21:17:12.637ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46912025-10-03T21:17:12.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46922025-10-03T21:17:12.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46932025-10-03T21:17:12.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46942025-10-03T21:17:12.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46952025-10-03T21:17:12.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46962025-10-03T21:17:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46972025-10-03T21:17:12.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46982025-10-03T21:17:12.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46992025-10-03T21:17:12.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47002025-10-03T21:17:12.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47012025-10-03T21:17:12.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47022025-10-03T21:17:12.650ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
47032025-10-03T21:17:12.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47042025-10-03T21:17:12.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47052025-10-03T21:17:12.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47062025-10-03T21:17:12.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47072025-10-03T21:17:12.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47082025-10-03T21:17:12.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47092025-10-03T21:17:12.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47102025-10-03T21:17:12.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
47112025-10-03T21:17:12.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
47122025-10-03T21:17:12.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
47132025-10-03T21:17:12.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47142025-10-03T21:17:12.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
47152025-10-03T21:17:12.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
47162025-10-03T21:17:12.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
47172025-10-03T21:17:12.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47182025-10-03T21:17:12.667ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
47192025-10-03T21:17:12.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
47202025-10-03T21:17:12.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
47212025-10-03T21:17:12.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
47222025-10-03T21:17:12.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
47232025-10-03T21:17:12.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47242025-10-03T21:17:12.672ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
47252025-10-03T21:17:12.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
47262025-10-03T21:17:12.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
47272025-10-03T21:17:12.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47282025-10-03T21:17:12.675ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
47292025-10-03T21:17:12.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
47302025-10-03T21:17:12.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
47312025-10-03T21:17:12.676ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
47322025-10-03T21:17:12.677ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 241.622087ms iteration = 16 status = {"blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","inventory_collection_id":"8e77a9b0-59eb-492d-aaf5-16d924b6dcdc","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
47332025-10-03T21:17:12.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47342025-10-03T21:17:12.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47352025-10-03T21:17:12.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47362025-10-03T21:17:12.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47372025-10-03T21:17:12.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47382025-10-03T21:17:12.694ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
47392025-10-03T21:17:12.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47402025-10-03T21:17:12.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47412025-10-03T21:17:12.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47422025-10-03T21:17:12.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47432025-10-03T21:17:12.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47442025-10-03T21:17:12.702ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
47452025-10-03T21:17:12.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47462025-10-03T21:17:12.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47472025-10-03T21:17:12.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47482025-10-03T21:17:12.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47492025-10-03T21:17:12.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47502025-10-03T21:17:12.708ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
47512025-10-03T21:17:12.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47522025-10-03T21:17:12.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47532025-10-03T21:17:12.710ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47542025-10-03T21:17:12.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47552025-10-03T21:17:12.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47562025-10-03T21:17:12.713ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
47572025-10-03T21:17:12.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47582025-10-03T21:17:12.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47592025-10-03T21:17:12.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47602025-10-03T21:17:12.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47612025-10-03T21:17:12.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47622025-10-03T21:17:12.719ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
47632025-10-03T21:17:12.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47642025-10-03T21:17:12.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47652025-10-03T21:17:12.721ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
47662025-10-03T21:17:12.721ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
47672025-10-03T21:17:12.721ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = da32185a-4ee3-4f78-acff-68419e269e03 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
47682025-10-03T21:17:12.721ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
47692025-10-03T21:17:12.721ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 285.876196ms iteration = 26 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
47702025-10-03T21:17:15.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 122 reason = Timeout
47712025-10-03T21:17:15.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
47722025-10-03T21:17:15.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
47732025-10-03T21:17:15.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47742025-10-03T21:17:15.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
47752025-10-03T21:17:15.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
47762025-10-03T21:17:15.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
47772025-10-03T21:17:15.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
47782025-10-03T21:17:15.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.10807ms iteration = 122 status = {"config_updated":false}
47792025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
47802025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47812025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
47822025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 123 reason = Timeout
47832025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47842025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
47852025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47862025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
47872025-10-03T21:17:20.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
47882025-10-03T21:17:20.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47892025-10-03T21:17:20.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47902025-10-03T21:17:20.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47912025-10-03T21:17:20.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47922025-10-03T21:17:20.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.066896ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47932025-10-03T21:17:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
47942025-10-03T21:17:20.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
47952025-10-03T21:17:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
47962025-10-03T21:17:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
47972025-10-03T21:17:20.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47982025-10-03T21:17:20.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
47992025-10-03T21:17:20.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
48002025-10-03T21:17:20.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.280622ms iteration = 123 status = {"config_updated":false}
48012025-10-03T21:17:20.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48022025-10-03T21:17:20.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48032025-10-03T21:17:20.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48042025-10-03T21:17:20.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
48052025-10-03T21:17:20.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48062025-10-03T21:17:20.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48072025-10-03T21:17:20.218ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
48082025-10-03T21:17:20.218ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.862784ms iteration = 62 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
48092025-10-03T21:17:21.225ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 476 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 9882562e-183a-4e4e-84bc-ef44bc8f09db response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
48102025-10-03T21:17:25.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 124 reason = Timeout
48112025-10-03T21:17:25.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
48122025-10-03T21:17:25.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
48132025-10-03T21:17:25.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48142025-10-03T21:17:25.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
48152025-10-03T21:17:25.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
48162025-10-03T21:17:25.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
48172025-10-03T21:17:25.197ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
48182025-10-03T21:17:25.197ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.046528ms iteration = 124 status = {"config_updated":false}
48192025-10-03T21:17:30.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
48202025-10-03T21:17:30.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48212025-10-03T21:17:30.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
48222025-10-03T21:17:30.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48232025-10-03T21:17:30.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 125 reason = Timeout
48242025-10-03T21:17:30.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
48252025-10-03T21:17:30.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48262025-10-03T21:17:30.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48272025-10-03T21:17:30.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
48282025-10-03T21:17:30.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48292025-10-03T21:17:30.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48302025-10-03T21:17:30.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48312025-10-03T21:17:30.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48322025-10-03T21:17:30.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.812003ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
48332025-10-03T21:17:30.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
48342025-10-03T21:17:30.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
48352025-10-03T21:17:30.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48362025-10-03T21:17:30.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
48372025-10-03T21:17:30.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48382025-10-03T21:17:30.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48392025-10-03T21:17:30.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48402025-10-03T21:17:30.199ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
48412025-10-03T21:17:30.199ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.497475ms iteration = 125 status = {"config_updated":false}
48422025-10-03T21:17:30.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48432025-10-03T21:17:30.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48442025-10-03T21:17:30.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
48452025-10-03T21:17:30.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48462025-10-03T21:17:30.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48472025-10-03T21:17:30.214ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
48482025-10-03T21:17:30.214ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.949249ms iteration = 63 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
48492025-10-03T21:17:31.225ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 442 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 59d1022a-8cf1-4903-a347-f606bbbe515f response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
48502025-10-03T21:17:35.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 126 reason = Timeout
48512025-10-03T21:17:35.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
48522025-10-03T21:17:35.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
48532025-10-03T21:17:35.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48542025-10-03T21:17:35.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
48552025-10-03T21:17:35.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
48562025-10-03T21:17:35.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
48572025-10-03T21:17:35.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
48582025-10-03T21:17:35.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.988339ms iteration = 126 status = {"config_updated":false}
48592025-10-03T21:17:36.217ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = c90b640a-3fb9-4cc8-bada-e0208fec9407 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
48602025-10-03T21:17:36.217ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = c90b640a-3fb9-4cc8-bada-e0208fec9407 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
48612025-10-03T21:17:36.220ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3216 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = c90b640a-3fb9-4cc8-bada-e0208fec9407 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending
48622025-10-03T21:17:36.220ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 715cae7f-d6ea-4d4d-9ba2-bcd0f210a76a roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
48632025-10-03T21:17:36.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 715cae7f-d6ea-4d4d-9ba2-bcd0f210a76a resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
48642025-10-03T21:17:36.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2697 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37452 req_id = 715cae7f-d6ea-4d4d-9ba2-bcd0f210a76a response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D
48652025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
48662025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
48672025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
48682025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 22 reason = Timeout
48692025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
48702025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
48712025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
48722025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 23.461\u{b5}s iteration = 22 status = {"disabled":true,"errors":[],"sps":[]}
48732025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
48742025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
48752025-10-03T21:17:40.191ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
48762025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48772025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
48782025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48792025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
48802025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48812025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48822025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
48832025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48842025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
48852025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 33 reason = Timeout
48862025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48872025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 127 reason = Timeout
48882025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
48892025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
48902025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
48912025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
48922025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48932025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
48942025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
48952025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
48962025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
48972025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
48982025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Timeout
48992025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
49002025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49012025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
49022025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
49032025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49042025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
49052025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49062025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
49072025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
49082025-10-03T21:17:40.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
49092025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
49102025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
49112025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49122025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
49132025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
49142025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
49152025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49162025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49172025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
49182025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
49192025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
49202025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49212025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
49222025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
49232025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49242025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
49252025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
49262025-10-03T21:17:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
49272025-10-03T21:17:40.194ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
49282025-10-03T21:17:40.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49292025-10-03T21:17:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
49302025-10-03T21:17:40.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49312025-10-03T21:17:40.195ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49322025-10-03T21:17:40.195ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49332025-10-03T21:17:40.195ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49342025-10-03T21:17:40.195ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49352025-10-03T21:17:40.196ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5105, tv_nsec: 684750270 } })
49362025-10-03T21:17:40.197ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
49372025-10-03T21:17:40.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.272331ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
49382025-10-03T21:17:40.197ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5105, tv_nsec: 686172586 } })
49392025-10-03T21:17:40.198ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.880387ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
49402025-10-03T21:17:40.198ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
49412025-10-03T21:17:40.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49422025-10-03T21:17:40.200ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49432025-10-03T21:17:40.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49442025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
49452025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49462025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
49472025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
49482025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49492025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
49502025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
49512025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49522025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
49532025-10-03T21:17:40.201ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
49542025-10-03T21:17:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
49552025-10-03T21:17:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49562025-10-03T21:17:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
49572025-10-03T21:17:40.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49582025-10-03T21:17:40.204ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49592025-10-03T21:17:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49602025-10-03T21:17:40.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49612025-10-03T21:17:40.206ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49622025-10-03T21:17:40.207ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 16.158532ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
49632025-10-03T21:17:40.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49642025-10-03T21:17:40.209ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
49652025-10-03T21:17:40.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49662025-10-03T21:17:40.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
49672025-10-03T21:17:40.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49682025-10-03T21:17:40.210ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49692025-10-03T21:17:40.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
49702025-10-03T21:17:40.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
49712025-10-03T21:17:40.210ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49722025-10-03T21:17:40.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49732025-10-03T21:17:40.210ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
49742025-10-03T21:17:40.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
49752025-10-03T21:17:40.212ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
49762025-10-03T21:17:40.213ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 21.532983ms iteration = 127 status = {"config_updated":false}
49772025-10-03T21:17:40.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
49782025-10-03T21:17:40.213ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49792025-10-03T21:17:40.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49802025-10-03T21:17:40.214ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 22.585685ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
49812025-10-03T21:17:40.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49822025-10-03T21:17:40.216ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5105, tv_nsec: 704738895 } })
49832025-10-03T21:17:40.216ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
49842025-10-03T21:17:40.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
49852025-10-03T21:17:40.218ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
49862025-10-03T21:17:40.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
49872025-10-03T21:17:40.270ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 79.438637ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
49882025-10-03T21:17:40.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 88.896533ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
49892025-10-03T21:17:40.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49902025-10-03T21:17:40.285ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
49912025-10-03T21:17:40.285ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49922025-10-03T21:17:40.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
49932025-10-03T21:17:40.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49942025-10-03T21:17:40.287ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49952025-10-03T21:17:40.287ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
49962025-10-03T21:17:40.287ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49972025-10-03T21:17:40.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
49982025-10-03T21:17:40.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 96.60769ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49992025-10-03T21:17:40.288ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
50002025-10-03T21:17:40.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50012025-10-03T21:17:40.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50022025-10-03T21:17:40.288ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 97.190337ms iteration = 33 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
50032025-10-03T21:17:40.288ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 97.250598ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
50042025-10-03T21:17:40.288ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 97.312498ms iteration = 35 status = {"physical_disks_added":0}
50052025-10-03T21:17:40.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
50062025-10-03T21:17:40.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50072025-10-03T21:17:40.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
50082025-10-03T21:17:40.291ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50092025-10-03T21:17:40.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50102025-10-03T21:17:40.292ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50112025-10-03T21:17:40.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50122025-10-03T21:17:40.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
50132025-10-03T21:17:40.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50142025-10-03T21:17:40.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50152025-10-03T21:17:40.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50162025-10-03T21:17:40.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50172025-10-03T21:17:40.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
50182025-10-03T21:17:40.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50192025-10-03T21:17:40.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
50202025-10-03T21:17:40.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
50212025-10-03T21:17:40.297ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50222025-10-03T21:17:40.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
50232025-10-03T21:17:40.297ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
50242025-10-03T21:17:40.297ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50252025-10-03T21:17:40.298ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1595376f-1f9c-4254-acb1-6828478a4755", "content-length": "26", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
50262025-10-03T21:17:40.298ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50272025-10-03T21:17:40.299ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
50282025-10-03T21:17:40.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
50292025-10-03T21:17:40.299ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50302025-10-03T21:17:40.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50312025-10-03T21:17:40.300ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
50322025-10-03T21:17:40.300ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50332025-10-03T21:17:40.301ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6c163f2-ce79-4628-93c0-affa74be3b3c", "content-length": "26", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
50342025-10-03T21:17:40.301ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50352025-10-03T21:17:40.302ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 111.028924ms iteration = 22 status = 0
50362025-10-03T21:17:40.303ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
50372025-10-03T21:17:40.305ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
50382025-10-03T21:17:40.305ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 113.520291ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
50392025-10-03T21:17:40.313ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
50402025-10-03T21:17:40.313ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 121.689254ms iteration = 64 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
50412025-10-03T21:17:40.335ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
50422025-10-03T21:17:40.335ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
50432025-10-03T21:17:40.335ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
50442025-10-03T21:17:40.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
50452025-10-03T21:17:40.340ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
50462025-10-03T21:17:40.340ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50472025-10-03T21:17:40.341ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f6f0fc0-81d5-4201-ac98-a23fda5e2b06", "content-length": "372", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
50482025-10-03T21:17:40.341ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50492025-10-03T21:17:40.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50502025-10-03T21:17:40.349ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
50512025-10-03T21:17:40.349ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50522025-10-03T21:17:40.349ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fdaa652-4024-4c2e-b47d-878a6003b84d", "content-length": "26", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
50532025-10-03T21:17:40.349ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50542025-10-03T21:17:40.350ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50552025-10-03T21:17:40.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
50562025-10-03T21:17:40.351ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
50572025-10-03T21:17:40.353ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50582025-10-03T21:17:40.355ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
50592025-10-03T21:17:40.355ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50602025-10-03T21:17:40.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
50612025-10-03T21:17:40.357ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50622025-10-03T21:17:40.359ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
50632025-10-03T21:17:40.359ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
50642025-10-03T21:17:40.359ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1d1acdc0-241e-4dfc-b9e3-d9f82947b07d", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
50652025-10-03T21:17:40.359ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50662025-10-03T21:17:40.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50672025-10-03T21:17:40.360ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
50682025-10-03T21:17:40.360ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50692025-10-03T21:17:40.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
50702025-10-03T21:17:40.361ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50712025-10-03T21:17:40.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50722025-10-03T21:17:40.365ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50732025-10-03T21:17:40.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50742025-10-03T21:17:40.368ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
50752025-10-03T21:17:40.368ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50762025-10-03T21:17:40.368ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ae675421-6045-4e2a-9f2a-217c8b777af2", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
50772025-10-03T21:17:40.369ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 178.593897ms iteration = 22 status = {}
50782025-10-03T21:17:40.369ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
50792025-10-03T21:17:40.369ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
50802025-10-03T21:17:40.369ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 178.393714ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
50812025-10-03T21:17:40.413ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
50822025-10-03T21:17:40.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
50832025-10-03T21:17:40.415ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 224.129251ms iteration = 22 status = null
50842025-10-03T21:17:40.436ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 245.555522ms iteration = 22 status = {}
50852025-10-03T21:17:40.477ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
50862025-10-03T21:17:40.477ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7111fdf-ff8c-4b1d-83db-fd1e4d3600a6", "content-length": "691", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
50872025-10-03T21:17:40.477ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
50882025-10-03T21:17:40.477ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
50892025-10-03T21:17:40.477ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
50902025-10-03T21:17:40.479ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 287.820369ms iteration = 22 status = {}
50912025-10-03T21:17:40.523ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50922025-10-03T21:17:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50932025-10-03T21:17:40.526ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
50942025-10-03T21:17:40.526ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50952025-10-03T21:17:40.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50962025-10-03T21:17:40.537ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
50972025-10-03T21:17:40.537ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
50982025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
50992025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
51002025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51012025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51022025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51032025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
51042025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51052025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
51062025-10-03T21:17:40.538ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
51072025-10-03T21:17:40.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51082025-10-03T21:17:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51092025-10-03T21:17:40.541ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
51102025-10-03T21:17:40.541ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
51112025-10-03T21:17:40.649ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "acfd4452-c5ca-4eb8-8a55-68dfa4e83d4d", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
51122025-10-03T21:17:40.649ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
51132025-10-03T21:17:40.649ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
51142025-10-03T21:17:40.656ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a4749e8-e5c9-47ae-a4ca-abf031efefdf", "content-length": "682", "date": "Fri, 03 Oct 2025 21:17:40 GMT"} })
51152025-10-03T21:17:40.656ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51162025-10-03T21:17:40.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51172025-10-03T21:17:40.660ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51182025-10-03T21:17:40.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51192025-10-03T21:17:40.663ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51202025-10-03T21:17:40.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51212025-10-03T21:17:40.666ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51222025-10-03T21:17:40.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51232025-10-03T21:17:40.673ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 481.846929ms iteration = 22 status = {}
51242025-10-03T21:17:41.225ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 483 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 34efa8f2-8a65-4842-9ea9-248467897ae0 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
51252025-10-03T21:17:45.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 128 reason = Timeout
51262025-10-03T21:17:45.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51272025-10-03T21:17:45.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51282025-10-03T21:17:45.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51292025-10-03T21:17:45.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51302025-10-03T21:17:45.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51312025-10-03T21:17:45.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51322025-10-03T21:17:45.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
51332025-10-03T21:17:45.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.756096ms iteration = 128 status = {"config_updated":false}
51342025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
51352025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51362025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
51372025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 129 reason = Timeout
51382025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51392025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51402025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51412025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51422025-10-03T21:17:50.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51432025-10-03T21:17:50.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51442025-10-03T21:17:50.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51452025-10-03T21:17:50.194ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51462025-10-03T21:17:50.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51472025-10-03T21:17:50.196ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.870654ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51482025-10-03T21:17:50.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51492025-10-03T21:17:50.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51502025-10-03T21:17:50.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51512025-10-03T21:17:50.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51522025-10-03T21:17:50.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51532025-10-03T21:17:50.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51542025-10-03T21:17:50.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51552025-10-03T21:17:50.199ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
51562025-10-03T21:17:50.199ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.423483ms iteration = 129 status = {"config_updated":false}
51572025-10-03T21:17:50.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51582025-10-03T21:17:50.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51592025-10-03T21:17:50.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
51602025-10-03T21:17:50.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51612025-10-03T21:17:50.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51622025-10-03T21:17:50.219ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
51632025-10-03T21:17:50.219ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.066666ms iteration = 65 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
51642025-10-03T21:17:51.225ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 382 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 504bdc90-f13a-4417-afc6-bb522b9a28fd response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
51652025-10-03T21:17:55.191ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 130 reason = Timeout
51662025-10-03T21:17:55.191ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51672025-10-03T21:17:55.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51682025-10-03T21:17:55.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51692025-10-03T21:17:55.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51702025-10-03T21:17:55.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51712025-10-03T21:17:55.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51722025-10-03T21:17:55.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
51732025-10-03T21:17:55.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.006159ms iteration = 130 status = {"config_updated":false}
51742025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
51752025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51762025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
51772025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51782025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 131 reason = Timeout
51792025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51802025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51812025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51822025-10-03T21:18:00.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51832025-10-03T21:18:00.194ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51842025-10-03T21:18:00.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51852025-10-03T21:18:00.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51862025-10-03T21:18:00.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51872025-10-03T21:18:00.196ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.712362ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51882025-10-03T21:18:00.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51892025-10-03T21:18:00.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51902025-10-03T21:18:00.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51912025-10-03T21:18:00.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51922025-10-03T21:18:00.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51932025-10-03T21:18:00.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51942025-10-03T21:18:00.199ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
51952025-10-03T21:18:00.199ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.279102ms iteration = 131 status = {"config_updated":false}
51962025-10-03T21:18:00.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51972025-10-03T21:18:00.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51982025-10-03T21:18:00.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51992025-10-03T21:18:00.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
52002025-10-03T21:18:00.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52012025-10-03T21:18:00.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
52022025-10-03T21:18:00.215ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
52032025-10-03T21:18:00.215ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.689247ms iteration = 66 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
52042025-10-03T21:18:01.225ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 457 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 4018482a-3ca6-4957-95aa-0a2efd5a2638 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
52052025-10-03T21:18:05.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 132 reason = Timeout
52062025-10-03T21:18:05.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52072025-10-03T21:18:05.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52082025-10-03T21:18:05.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52092025-10-03T21:18:05.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
52102025-10-03T21:18:05.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52112025-10-03T21:18:05.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52122025-10-03T21:18:05.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
52132025-10-03T21:18:05.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.039968ms iteration = 132 status = {"config_updated":false}
52142025-10-03T21:18:10.191ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
52152025-10-03T21:18:10.191ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
52162025-10-03T21:18:10.191ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
52172025-10-03T21:18:10.191ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
52182025-10-03T21:18:10.191ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
52192025-10-03T21:18:10.191ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
52202025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
52212025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
52222025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
52232025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
52242025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
52252025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
52262025-10-03T21:18:10.192ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
52272025-10-03T21:18:10.192ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = ExternalNetworking step id = Ensure
52282025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
52292025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52302025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
52312025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
52322025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52332025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
52342025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
52352025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
52362025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52372025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
52382025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52392025-10-03T21:18:10.192ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-10-03 21:08:10.192149623 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
52402025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
52412025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
52422025-10-03T21:18:10.192ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
52432025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
52442025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
52452025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52462025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
52472025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 36 reason = Timeout
52482025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 133 reason = Timeout
52492025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
52502025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52512025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52522025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52532025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
52542025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52552025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52562025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
52572025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
52582025-10-03T21:18:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52592025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
52602025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
52612025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
52622025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52632025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 27 reason = Timeout
52642025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52652025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52662025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
52672025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
52682025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52692025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
52702025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52712025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
52722025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
52732025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
52742025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52752025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
52762025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
52772025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
52782025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
52792025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
52802025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
52812025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52822025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
52832025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
52842025-10-03T21:18:10.193ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
52852025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
52862025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
52872025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52882025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
52892025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
52902025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
52912025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.8\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"sps":[]}
52922025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
52932025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52942025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
52952025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52962025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52972025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
52982025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 34 reason = Timeout
52992025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
53002025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
53012025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
53022025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53032025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53042025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
53052025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
53062025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
53072025-10-03T21:18:10.193ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("486849a9-1733-4aa9-9b66-43ace2da0eea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 5135, tv_nsec: 680952507 } })
53082025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
53092025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
53102025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53112025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
53122025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
53132025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
53142025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
53152025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
53162025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
53172025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
53182025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
53192025-10-03T21:18:10.193ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2efcfb05-0c3d-45aa-b120-c334f3706be4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a7c1656-d513-4cc3-b2b0-8e31ad2b5545.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 5135, tv_nsec: 681280210 } })
53202025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
53212025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53222025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
53232025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
53242025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53252025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
53262025-10-03T21:18:10.193ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.151194ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
53272025-10-03T21:18:10.194ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
53282025-10-03T21:18:10.194ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.392367ms iteration = 12 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
53292025-10-03T21:18:10.196ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53302025-10-03T21:18:10.196ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53312025-10-03T21:18:10.196ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53322025-10-03T21:18:10.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53332025-10-03T21:18:10.196ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.767122ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
53342025-10-03T21:18:10.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
53352025-10-03T21:18:10.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.968326ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
53362025-10-03T21:18:10.198ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5135, tv_nsec: 685658680 } })
53372025-10-03T21:18:10.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
53382025-10-03T21:18:10.198ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53392025-10-03T21:18:10.198ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5135, tv_nsec: 686036284 } })
53402025-10-03T21:18:10.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53412025-10-03T21:18:10.198ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
53422025-10-03T21:18:10.198ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
53432025-10-03T21:18:10.198ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53442025-10-03T21:18:10.199ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53452025-10-03T21:18:10.199ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53462025-10-03T21:18:10.199ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.042461ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
53472025-10-03T21:18:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53482025-10-03T21:18:10.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
53492025-10-03T21:18:10.200ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53502025-10-03T21:18:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
53512025-10-03T21:18:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
53522025-10-03T21:18:10.201ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53532025-10-03T21:18:10.201ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-10-03 21:08:10.192149623 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
53542025-10-03T21:18:10.201ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.403176ms iteration = 12 status = {"expiration":"2025-10-03T21:08:10.192149623Z","pruned":[]}
53552025-10-03T21:18:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
53562025-10-03T21:18:10.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53572025-10-03T21:18:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53582025-10-03T21:18:10.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53592025-10-03T21:18:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
53602025-10-03T21:18:10.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53612025-10-03T21:18:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53622025-10-03T21:18:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53632025-10-03T21:18:10.204ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
53642025-10-03T21:18:10.204ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 12.026456ms iteration = 12 status = {"by_rx":{},"error":null}
53652025-10-03T21:18:10.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53662025-10-03T21:18:10.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
53672025-10-03T21:18:10.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53682025-10-03T21:18:10.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53692025-10-03T21:18:10.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53702025-10-03T21:18:10.210ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53712025-10-03T21:18:10.213ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 20.831755ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53722025-10-03T21:18:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
53732025-10-03T21:18:10.214ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
53742025-10-03T21:18:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
53752025-10-03T21:18:10.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
53762025-10-03T21:18:10.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53772025-10-03T21:18:10.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
53782025-10-03T21:18:10.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53792025-10-03T21:18:10.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
53802025-10-03T21:18:10.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53812025-10-03T21:18:10.218ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53822025-10-03T21:18:10.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53832025-10-03T21:18:10.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53842025-10-03T21:18:10.218ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53852025-10-03T21:18:10.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53862025-10-03T21:18:10.223ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
53872025-10-03T21:18:10.223ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53882025-10-03T21:18:10.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53892025-10-03T21:18:10.224ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53902025-10-03T21:18:10.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
53912025-10-03T21:18:10.226ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53922025-10-03T21:18:10.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
53932025-10-03T21:18:10.226ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
53942025-10-03T21:18:10.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
53952025-10-03T21:18:10.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
53962025-10-03T21:18:10.227ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
53972025-10-03T21:18:10.227ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
53982025-10-03T21:18:10.227ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 34.436098ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
53992025-10-03T21:18:10.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
54002025-10-03T21:18:10.228ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54012025-10-03T21:18:10.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
54022025-10-03T21:18:10.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54032025-10-03T21:18:10.230ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54042025-10-03T21:18:10.230ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 36.999008ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"195.0.0","globs_reprocessed":{},"no_receivers":[]}
54052025-10-03T21:18:10.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54062025-10-03T21:18:10.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54072025-10-03T21:18:10.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
54082025-10-03T21:18:10.238ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
54092025-10-03T21:18:10.238ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
54102025-10-03T21:18:10.238ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
54112025-10-03T21:18:10.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54122025-10-03T21:18:10.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54132025-10-03T21:18:10.240ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
54142025-10-03T21:18:10.240ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
54152025-10-03T21:18:10.240ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
54162025-10-03T21:18:10.240ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54172025-10-03T21:18:10.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54182025-10-03T21:18:10.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54192025-10-03T21:18:10.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54202025-10-03T21:18:10.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54212025-10-03T21:18:10.241ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
54222025-10-03T21:18:10.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
54232025-10-03T21:18:10.242ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 49.960754ms iteration = 133 status = {"config_updated":false}
54242025-10-03T21:18:10.242ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5135, tv_nsec: 730069421 } })
54252025-10-03T21:18:10.243ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
54262025-10-03T21:18:10.245ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54272025-10-03T21:18:10.246ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 53.634626ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
54282025-10-03T21:18:10.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
54292025-10-03T21:18:10.251ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54302025-10-03T21:18:10.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
54312025-10-03T21:18:10.253ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54322025-10-03T21:18:10.254ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54332025-10-03T21:18:10.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54342025-10-03T21:18:10.257ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54352025-10-03T21:18:10.258ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54362025-10-03T21:18:10.258ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
54372025-10-03T21:18:10.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54382025-10-03T21:18:10.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54392025-10-03T21:18:10.259ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54402025-10-03T21:18:10.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54412025-10-03T21:18:10.260ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54422025-10-03T21:18:10.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54432025-10-03T21:18:10.262ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
54442025-10-03T21:18:10.263ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54452025-10-03T21:18:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
54462025-10-03T21:18:10.264ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54472025-10-03T21:18:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54482025-10-03T21:18:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
54492025-10-03T21:18:10.265ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
54502025-10-03T21:18:10.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54512025-10-03T21:18:10.265ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54522025-10-03T21:18:10.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54532025-10-03T21:18:10.267ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54542025-10-03T21:18:10.268ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
54552025-10-03T21:18:10.268ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
54562025-10-03T21:18:10.268ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54572025-10-03T21:18:10.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
54582025-10-03T21:18:10.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54592025-10-03T21:18:10.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54602025-10-03T21:18:10.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
54612025-10-03T21:18:10.269ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
54622025-10-03T21:18:10.269ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
54632025-10-03T21:18:10.269ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54642025-10-03T21:18:10.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54652025-10-03T21:18:10.270ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54662025-10-03T21:18:10.271ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
54672025-10-03T21:18:10.271ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54682025-10-03T21:18:10.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
54692025-10-03T21:18:10.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54702025-10-03T21:18:10.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
54712025-10-03T21:18:10.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54722025-10-03T21:18:10.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54732025-10-03T21:18:10.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54742025-10-03T21:18:10.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54752025-10-03T21:18:10.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54762025-10-03T21:18:10.280ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 88.708171ms iteration = 36 status = {"physical_disks_added":0}
54772025-10-03T21:18:10.281ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
54782025-10-03T21:18:10.281ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
54792025-10-03T21:18:10.281ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-10-03 21:07:12.445842 UTC dns_group = internal generation = 1
54802025-10-03T21:18:10.281ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 89.130106ms iteration = 12 status = {"generation":1}
54812025-10-03T21:18:10.283ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
54822025-10-03T21:18:10.283ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
54832025-10-03T21:18:10.283ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-10-03 21:07:14.470425 UTC dns_group = external generation = 2
54842025-10-03T21:18:10.283ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 91.23591ms iteration = 12 status = {"generation":2}
54852025-10-03T21:18:10.283ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
54862025-10-03T21:18:10.293ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
54872025-10-03T21:18:10.298ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54882025-10-03T21:18:10.298ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
54892025-10-03T21:18:10.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54902025-10-03T21:18:10.301ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
54912025-10-03T21:18:10.304ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
54922025-10-03T21:18:10.307ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
54932025-10-03T21:18:10.309ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
54942025-10-03T21:18:10.317ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 124.485216ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
54952025-10-03T21:18:10.322ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
54962025-10-03T21:18:10.322ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
54972025-10-03T21:18:10.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
54982025-10-03T21:18:10.323ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
54992025-10-03T21:18:10.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
55002025-10-03T21:18:10.327ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
55012025-10-03T21:18:10.327ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
55022025-10-03T21:18:10.327ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 134.963594ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
55032025-10-03T21:18:10.333ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
55042025-10-03T21:18:10.333ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 140.92155ms iteration = 67 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
55052025-10-03T21:18:10.334ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
55062025-10-03T21:18:10.335ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55072025-10-03T21:18:10.335ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
55082025-10-03T21:18:10.335ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55092025-10-03T21:18:10.335ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = SledList step id = Fetch
55102025-10-03T21:18:10.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55112025-10-03T21:18:10.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
55122025-10-03T21:18:10.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55132025-10-03T21:18:10.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55142025-10-03T21:18:10.338ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 145.186889ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
55152025-10-03T21:18:10.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
55162025-10-03T21:18:10.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55172025-10-03T21:18:10.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55182025-10-03T21:18:10.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55192025-10-03T21:18:10.344ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55202025-10-03T21:18:10.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55212025-10-03T21:18:10.346ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
55222025-10-03T21:18:10.346ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
55232025-10-03T21:18:10.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55242025-10-03T21:18:10.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55252025-10-03T21:18:10.347ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55262025-10-03T21:18:10.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
55272025-10-03T21:18:10.347ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
55282025-10-03T21:18:10.347ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
55292025-10-03T21:18:10.347ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 154.618536ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
55302025-10-03T21:18:10.348ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 156.494347ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
55312025-10-03T21:18:10.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
55322025-10-03T21:18:10.350ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55332025-10-03T21:18:10.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55342025-10-03T21:18:10.353ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
55352025-10-03T21:18:10.353ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 160.095417ms iteration = 34 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
55362025-10-03T21:18:10.359ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = DeployNexusRecords step id = Ensure
55372025-10-03T21:18:10.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55382025-10-03T21:18:10.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55392025-10-03T21:18:10.361ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55402025-10-03T21:18:10.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55412025-10-03T21:18:10.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55422025-10-03T21:18:10.363ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
55432025-10-03T21:18:10.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55442025-10-03T21:18:10.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55452025-10-03T21:18:10.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55462025-10-03T21:18:10.365ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
55472025-10-03T21:18:10.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55482025-10-03T21:18:10.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
55492025-10-03T21:18:10.366ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55502025-10-03T21:18:10.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55512025-10-03T21:18:10.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55522025-10-03T21:18:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55532025-10-03T21:18:10.368ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 175.606442ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
55542025-10-03T21:18:10.368ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55552025-10-03T21:18:10.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55562025-10-03T21:18:10.372ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
55572025-10-03T21:18:10.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55582025-10-03T21:18:10.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55592025-10-03T21:18:10.373ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
55602025-10-03T21:18:10.373ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55612025-10-03T21:18:10.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55622025-10-03T21:18:10.374ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-10-03T21:07:04.185535Z, time_modified: 2025-10-03T21:07:04.185535Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }) }
55632025-10-03T21:18:10.374ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 182.038334ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]}},"default_endpoint":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
55642025-10-03T21:18:10.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55652025-10-03T21:18:10.376ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 183.842235ms iteration = 23 status = 0
55662025-10-03T21:18:10.378ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
55672025-10-03T21:18:10.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55682025-10-03T21:18:10.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55692025-10-03T21:18:10.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55702025-10-03T21:18:10.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55712025-10-03T21:18:10.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55722025-10-03T21:18:10.386ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
55732025-10-03T21:18:10.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55742025-10-03T21:18:10.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55752025-10-03T21:18:10.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55762025-10-03T21:18:10.391ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
55772025-10-03T21:18:10.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55782025-10-03T21:18:10.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55792025-10-03T21:18:10.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55802025-10-03T21:18:10.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55812025-10-03T21:18:10.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55822025-10-03T21:18:10.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55832025-10-03T21:18:10.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55842025-10-03T21:18:10.409ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
55852025-10-03T21:18:10.410ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55862025-10-03T21:18:10.410ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
55872025-10-03T21:18:10.410ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 217.99873ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
55882025-10-03T21:18:10.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
55892025-10-03T21:18:10.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55902025-10-03T21:18:10.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55912025-10-03T21:18:10.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55922025-10-03T21:18:10.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55932025-10-03T21:18:10.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55942025-10-03T21:18:10.422ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
55952025-10-03T21:18:10.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
55962025-10-03T21:18:10.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
55972025-10-03T21:18:10.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55982025-10-03T21:18:10.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55992025-10-03T21:18:10.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56002025-10-03T21:18:10.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56012025-10-03T21:18:10.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
56022025-10-03T21:18:10.431ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56032025-10-03T21:18:10.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56042025-10-03T21:18:10.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56052025-10-03T21:18:10.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56062025-10-03T21:18:10.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56072025-10-03T21:18:10.435ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
56082025-10-03T21:18:10.435ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56092025-10-03T21:18:10.436ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee8ad26e-16e5-40c8-86c4-a36f48c873e7", "content-length": "26", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} })
56102025-10-03T21:18:10.436ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56112025-10-03T21:18:10.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56122025-10-03T21:18:10.442ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
56132025-10-03T21:18:10.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56142025-10-03T21:18:10.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56152025-10-03T21:18:10.444ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
56162025-10-03T21:18:10.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56172025-10-03T21:18:10.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56182025-10-03T21:18:10.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56192025-10-03T21:18:10.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56202025-10-03T21:18:10.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56212025-10-03T21:18:10.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56222025-10-03T21:18:10.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56232025-10-03T21:18:10.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56242025-10-03T21:18:10.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56252025-10-03T21:18:10.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56262025-10-03T21:18:10.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56272025-10-03T21:18:10.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56282025-10-03T21:18:10.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56292025-10-03T21:18:10.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56302025-10-03T21:18:10.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56312025-10-03T21:18:10.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56322025-10-03T21:18:10.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56332025-10-03T21:18:10.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56342025-10-03T21:18:10.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56352025-10-03T21:18:10.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56362025-10-03T21:18:10.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56372025-10-03T21:18:10.476ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56382025-10-03T21:18:10.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56392025-10-03T21:18:10.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56402025-10-03T21:18:10.479ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56412025-10-03T21:18:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
56422025-10-03T21:18:10.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56432025-10-03T21:18:10.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56442025-10-03T21:18:10.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56452025-10-03T21:18:10.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56462025-10-03T21:18:10.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56472025-10-03T21:18:10.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
56482025-10-03T21:18:10.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56492025-10-03T21:18:10.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56502025-10-03T21:18:10.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56512025-10-03T21:18:10.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56522025-10-03T21:18:10.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56532025-10-03T21:18:10.498ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = SledAgent step id = Ensure
56542025-10-03T21:18:10.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
56552025-10-03T21:18:10.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56562025-10-03T21:18:10.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56572025-10-03T21:18:10.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56582025-10-03T21:18:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56592025-10-03T21:18:10.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56602025-10-03T21:18:10.506ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
56612025-10-03T21:18:10.506ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56622025-10-03T21:18:10.507ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "078869a3-a93d-4f26-b563-e581e7a491b0", "content-length": "26", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} })
56632025-10-03T21:18:10.507ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56642025-10-03T21:18:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
56652025-10-03T21:18:10.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56662025-10-03T21:18:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56672025-10-03T21:18:10.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56682025-10-03T21:18:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56692025-10-03T21:18:10.511ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56702025-10-03T21:18:10.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56712025-10-03T21:18:10.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
56722025-10-03T21:18:10.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56732025-10-03T21:18:10.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56742025-10-03T21:18:10.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56752025-10-03T21:18:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56762025-10-03T21:18:10.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56772025-10-03T21:18:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
56782025-10-03T21:18:10.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56792025-10-03T21:18:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56802025-10-03T21:18:10.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56812025-10-03T21:18:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
56822025-10-03T21:18:10.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56832025-10-03T21:18:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56842025-10-03T21:18:10.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56852025-10-03T21:18:10.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56862025-10-03T21:18:10.536ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
56872025-10-03T21:18:10.536ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56882025-10-03T21:18:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
56892025-10-03T21:18:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
56902025-10-03T21:18:10.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56912025-10-03T21:18:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56922025-10-03T21:18:10.539ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56932025-10-03T21:18:10.539ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
56942025-10-03T21:18:10.539ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
56952025-10-03T21:18:10.539ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = ade00c7b-b50c-40fe-97d5-86e98791ef78 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
56962025-10-03T21:18:10.539ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
56972025-10-03T21:18:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 347.092337ms iteration = 27 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
56982025-10-03T21:18:10.541ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
56992025-10-03T21:18:10.541ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57002025-10-03T21:18:10.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
57012025-10-03T21:18:10.544ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
57022025-10-03T21:18:10.610ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
57032025-10-03T21:18:10.610ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
57042025-10-03T21:18:10.610ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3da64fbc-50b3-43c3-8914-a3ea4c53004d", "content-length": "26", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} })
57052025-10-03T21:18:10.610ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
57062025-10-03T21:18:10.612ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
57072025-10-03T21:18:10.624ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
57082025-10-03T21:18:10.624ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57092025-10-03T21:18:10.624ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf76a4e4-4db0-4405-949e-d28823a7f6a7", "content-length": "372", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} })
57102025-10-03T21:18:10.624ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57112025-10-03T21:18:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57122025-10-03T21:18:10.652ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
57132025-10-03T21:18:10.652ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 459.643448ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
57142025-10-03T21:18:10.673ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
57152025-10-03T21:18:10.674ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "06e9b6f5-664d-499a-998d-99f7d93ce034", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
57162025-10-03T21:18:10.674ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
57172025-10-03T21:18:10.675ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7d6469d4-c562-4633-8dfd-897a1f2d320a", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
57182025-10-03T21:18:10.675ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
57192025-10-03T21:18:10.678ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
57202025-10-03T21:18:10.678ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 486.314349ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
57212025-10-03T21:18:10.684ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
57222025-10-03T21:18:10.684ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
57232025-10-03T21:18:10.684ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "426e3c82-d810-4f44-9226-3f7773aa21aa", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} })
57242025-10-03T21:18:10.684ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57252025-10-03T21:18:10.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57262025-10-03T21:18:10.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57272025-10-03T21:18:10.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57282025-10-03T21:18:10.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57292025-10-03T21:18:10.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57302025-10-03T21:18:10.693ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
57312025-10-03T21:18:10.693ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57322025-10-03T21:18:10.693ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "36953344-c982-495c-a54b-2aa34f97f771", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} })
57332025-10-03T21:18:10.696ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 503.852167ms iteration = 23 status = {}
57342025-10-03T21:18:10.733ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
57352025-10-03T21:18:10.734ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "7fd99300-8cb9-437b-8b26-3b930bacd59d", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} }) sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
57362025-10-03T21:18:10.737ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = FirewallRules step id = Ensure
57372025-10-03T21:18:10.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57382025-10-03T21:18:10.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57392025-10-03T21:18:10.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57402025-10-03T21:18:10.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57412025-10-03T21:18:10.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57422025-10-03T21:18:10.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57432025-10-03T21:18:10.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57442025-10-03T21:18:10.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57452025-10-03T21:18:10.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57462025-10-03T21:18:10.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57472025-10-03T21:18:10.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57482025-10-03T21:18:10.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57492025-10-03T21:18:10.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57502025-10-03T21:18:10.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
57512025-10-03T21:18:10.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57522025-10-03T21:18:10.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57532025-10-03T21:18:10.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57542025-10-03T21:18:10.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57552025-10-03T21:18:10.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57562025-10-03T21:18:10.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57572025-10-03T21:18:10.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57582025-10-03T21:18:10.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
57592025-10-03T21:18:10.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57602025-10-03T21:18:10.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57612025-10-03T21:18:10.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57622025-10-03T21:18:10.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57632025-10-03T21:18:10.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57642025-10-03T21:18:10.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57652025-10-03T21:18:10.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57662025-10-03T21:18:10.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57672025-10-03T21:18:10.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57682025-10-03T21:18:10.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57692025-10-03T21:18:10.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57702025-10-03T21:18:10.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57712025-10-03T21:18:10.768ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
57722025-10-03T21:18:10.768ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b36b0aa0-451b-4d7c-acf6-748c69f1d379", "content-length": "691", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} })
57732025-10-03T21:18:10.770ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
57742025-10-03T21:18:10.770ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
57752025-10-03T21:18:10.770ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
57762025-10-03T21:18:10.773ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 580.192919ms iteration = 23 status = {}
57772025-10-03T21:18:10.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57782025-10-03T21:18:10.773ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
57792025-10-03T21:18:10.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
57802025-10-03T21:18:10.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57812025-10-03T21:18:10.776ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 584.139113ms iteration = 23 status = null
57822025-10-03T21:18:10.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57832025-10-03T21:18:10.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
57842025-10-03T21:18:10.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57852025-10-03T21:18:10.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57862025-10-03T21:18:10.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57872025-10-03T21:18:10.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57882025-10-03T21:18:10.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57892025-10-03T21:18:10.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57902025-10-03T21:18:10.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57912025-10-03T21:18:10.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57922025-10-03T21:18:10.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57932025-10-03T21:18:10.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57942025-10-03T21:18:10.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57952025-10-03T21:18:10.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57962025-10-03T21:18:10.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57972025-10-03T21:18:10.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57982025-10-03T21:18:10.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57992025-10-03T21:18:10.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
58002025-10-03T21:18:10.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58012025-10-03T21:18:10.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58022025-10-03T21:18:10.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58032025-10-03T21:18:10.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58042025-10-03T21:18:10.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58052025-10-03T21:18:10.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58062025-10-03T21:18:10.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58072025-10-03T21:18:10.805ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
58082025-10-03T21:18:10.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58092025-10-03T21:18:10.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58102025-10-03T21:18:10.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58112025-10-03T21:18:10.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58122025-10-03T21:18:10.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58132025-10-03T21:18:10.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58142025-10-03T21:18:10.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58152025-10-03T21:18:10.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58162025-10-03T21:18:10.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58172025-10-03T21:18:10.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58182025-10-03T21:18:10.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58192025-10-03T21:18:10.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58202025-10-03T21:18:10.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58212025-10-03T21:18:10.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58222025-10-03T21:18:10.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58232025-10-03T21:18:10.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58242025-10-03T21:18:10.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58252025-10-03T21:18:10.825ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
58262025-10-03T21:18:10.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58272025-10-03T21:18:10.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58282025-10-03T21:18:10.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58292025-10-03T21:18:10.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58302025-10-03T21:18:10.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58312025-10-03T21:18:10.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58322025-10-03T21:18:10.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58332025-10-03T21:18:10.833ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
58342025-10-03T21:18:10.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58352025-10-03T21:18:10.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58362025-10-03T21:18:10.839ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
58372025-10-03T21:18:10.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58382025-10-03T21:18:10.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58392025-10-03T21:18:10.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58402025-10-03T21:18:10.843ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
58412025-10-03T21:18:10.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58422025-10-03T21:18:10.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58432025-10-03T21:18:10.845ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
58442025-10-03T21:18:10.845ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
58452025-10-03T21:18:10.845ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
58462025-10-03T21:18:10.858ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 666.449692ms iteration = 23 status = {}
58472025-10-03T21:18:10.859ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
58482025-10-03T21:18:10.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58492025-10-03T21:18:10.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58502025-10-03T21:18:10.877ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
58512025-10-03T21:18:10.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58522025-10-03T21:18:10.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58532025-10-03T21:18:10.889ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
58542025-10-03T21:18:10.889ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
58552025-10-03T21:18:10.889ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
58562025-10-03T21:18:10.889ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
58572025-10-03T21:18:10.889ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58582025-10-03T21:18:10.889ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58592025-10-03T21:18:10.889ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58602025-10-03T21:18:10.889ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
58612025-10-03T21:18:10.890ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58622025-10-03T21:18:10.890ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
58632025-10-03T21:18:10.890ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
58642025-10-03T21:18:10.890ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58652025-10-03T21:18:10.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58662025-10-03T21:18:10.894ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
58672025-10-03T21:18:10.894ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
58682025-10-03T21:18:10.900ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), time_created: 2025-10-03T21:07:05.994Z, time_modified: 2025-10-03T21:07:05.994Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
58692025-10-03T21:18:10.900ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
58702025-10-03T21:18:10.901ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
58712025-10-03T21:18:10.901ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "3c1965c7-10ba-4d2a-8751-d2f33335f147", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
58722025-10-03T21:18:10.901ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
58732025-10-03T21:18:10.902ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "095df490-3058-4c73-b5cc-bb549502ab44", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
58742025-10-03T21:18:10.902ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
58752025-10-03T21:18:10.904ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "0401baba-4619-4821-bafa-6d27cc1bb1c3", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
58762025-10-03T21:18:10.904ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
58772025-10-03T21:18:10.906ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
58782025-10-03T21:18:10.906ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 714.441135ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
58792025-10-03T21:18:10.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58802025-10-03T21:18:10.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58812025-10-03T21:18:10.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58822025-10-03T21:18:10.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58832025-10-03T21:18:10.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58842025-10-03T21:18:10.939ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
58852025-10-03T21:18:10.939ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
58862025-10-03T21:18:10.940ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "8829e0fd-812f-4cda-b19a-9816c54e077b", "date": "Fri, 03 Oct 2025 21:18:10 GMT"} })
58872025-10-03T21:18:10.942ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
58882025-10-03T21:18:10.942ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = Dns step id = Ensure
58892025-10-03T21:18:10.942ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
58902025-10-03T21:18:10.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58912025-10-03T21:18:10.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58922025-10-03T21:18:10.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58932025-10-03T21:18:10.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58942025-10-03T21:18:10.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58952025-10-03T21:18:10.948ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
58962025-10-03T21:18:10.949ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
58972025-10-03T21:18:10.949ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
58982025-10-03T21:18:10.949ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
58992025-10-03T21:18:10.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59002025-10-03T21:18:10.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59012025-10-03T21:18:10.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59022025-10-03T21:18:10.954ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59032025-10-03T21:18:10.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59042025-10-03T21:18:10.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59052025-10-03T21:18:10.956ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
59062025-10-03T21:18:10.956ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
59072025-10-03T21:18:10.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59082025-10-03T21:18:10.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59092025-10-03T21:18:10.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59102025-10-03T21:18:10.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59112025-10-03T21:18:10.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59122025-10-03T21:18:10.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59132025-10-03T21:18:10.963ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
59142025-10-03T21:18:10.963ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
59152025-10-03T21:18:10.963ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
59162025-10-03T21:18:10.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59172025-10-03T21:18:10.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59182025-10-03T21:18:10.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59192025-10-03T21:18:10.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59202025-10-03T21:18:10.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59212025-10-03T21:18:10.969ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59222025-10-03T21:18:10.970ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
59232025-10-03T21:18:10.970ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
59242025-10-03T21:18:10.970ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
59252025-10-03T21:18:10.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59262025-10-03T21:18:10.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59272025-10-03T21:18:10.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59282025-10-03T21:18:10.975ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59292025-10-03T21:18:10.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59302025-10-03T21:18:10.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59312025-10-03T21:18:10.977ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
59322025-10-03T21:18:10.977ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
59332025-10-03T21:18:10.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59342025-10-03T21:18:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59352025-10-03T21:18:10.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59362025-10-03T21:18:10.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59372025-10-03T21:18:10.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59382025-10-03T21:18:10.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59392025-10-03T21:18:10.987ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
59402025-10-03T21:18:10.987ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
59412025-10-03T21:18:10.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59422025-10-03T21:18:10.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59432025-10-03T21:18:10.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59442025-10-03T21:18:10.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
59452025-10-03T21:18:10.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59462025-10-03T21:18:10.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59472025-10-03T21:18:10.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59482025-10-03T21:18:10.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59492025-10-03T21:18:10.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59502025-10-03T21:18:10.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59512025-10-03T21:18:10.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59522025-10-03T21:18:10.999ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59532025-10-03T21:18:11.001ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
59542025-10-03T21:18:11.001ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
59552025-10-03T21:18:11.001ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = OmicronZones step id = Cleanup
59562025-10-03T21:18:11.001ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = OmicronZones step id = Cleanup
59572025-10-03T21:18:11.001ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = PhysicalDisks step id = Cleanup
59582025-10-03T21:18:11.001ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = Clickhouse step id = Ensure
59592025-10-03T21:18:11.001ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = Clickhouse step id = Ensure
59602025-10-03T21:18:11.038ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
59612025-10-03T21:18:11.043ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "4709b33e-3aed-456a-8825-e3fcda1d68c5", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
59622025-10-03T21:18:11.044ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = SupportBundles step id = Cleanup
59632025-10-03T21:18:11.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59642025-10-03T21:18:11.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59652025-10-03T21:18:11.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59662025-10-03T21:18:11.048ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
59672025-10-03T21:18:11.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59682025-10-03T21:18:11.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59692025-10-03T21:18:11.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59702025-10-03T21:18:11.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59712025-10-03T21:18:11.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59722025-10-03T21:18:11.055ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
59732025-10-03T21:18:11.060ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = OmicronZones step id = Cleanup
59742025-10-03T21:18:11.060ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
59752025-10-03T21:18:11.060ZDEBGnexus (BackgroundTasks): re-assign sagas: found Nexus instances background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
59762025-10-03T21:18:11.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59772025-10-03T21:18:11.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59782025-10-03T21:18:11.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59792025-10-03T21:18:11.063ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
59802025-10-03T21:18:11.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59812025-10-03T21:18:11.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59822025-10-03T21:18:11.064ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:54 nexus_zone_ids = []
59832025-10-03T21:18:11.064ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(675bea75-8e2d-4986-86f2-b590a30bb084 (blueprint), false)
59842025-10-03T21:18:11.064ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = Cockroach step id = Ensure
59852025-10-03T21:18:11.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59862025-10-03T21:18:11.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59872025-10-03T21:18:11.083ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
59882025-10-03T21:18:11.083ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 025a20d6-7795-4af0-8045-a7ed99a7784d step component = MgsUpdates step id = Ensure
59892025-10-03T21:18:11.083ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Dependency
59902025-10-03T21:18:11.083ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 35 reason = Dependency
59912025-10-03T21:18:11.083ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59922025-10-03T21:18:11.083ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59932025-10-03T21:18:11.083ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 891.728715ms iteration = 13 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4140,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":143094605,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":143094555,"secs":0}},"event_index":1,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":143112546,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":24288644,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":24288595,"secs":0}},"event_index":2,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":167420621,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":138627675,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":138627645,"secs":0}},"event_index":3,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":306070056,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":238945786,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":238945746,"secs":0}},"event_index":4,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":545030372,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":205861033,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":205860993,"secs":0}},"event_index":5,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":750902796,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":58771084,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":58771034,"secs":0}},"event_index":6,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":809683280,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5390,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5350,"secs":0}},"event_index":7,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":809706090,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3260,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3210,"secs":0}},"event_index":8,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":809712800,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3600,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3560,"secs":0}},"event_index":9,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":809718660,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2240,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2190,"secs":0}},"event_index":10,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":809727040,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":42978245,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":42978174,"secs":0}},"event_index":11,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":852707085,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":15403634,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":15403594,"secs":0}},"event_index":12,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":868122890,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4630122,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4630082,"secs":0}},"event_index":13,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":872767762,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":18511529,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":18511489,"secs":0}},"event_index":14,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":891290641,"secs":0}},{"data":{"attempt_elapsed":{"nanos":5050,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5020,"secs":0}},"event_index":15,"execution_id":"a725871f-282a-4ad2-8d4a-6760270d624f","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":891306291,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"675bea75-8e2d-4986-86f2-b590a30bb084"}
59942025-10-03T21:18:11.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59952025-10-03T21:18:11.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
59962025-10-03T21:18:11.084ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8de90662-cb5f-4df0-a8d2-7bd440eb6967", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
59972025-10-03T21:18:11.084ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
59982025-10-03T21:18:11.084ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
59992025-10-03T21:18:11.085ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60002025-10-03T21:18:11.085ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60012025-10-03T21:18:11.088ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60022025-10-03T21:18:11.088ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60032025-10-03T21:18:11.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60042025-10-03T21:18:11.089ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
60052025-10-03T21:18:11.089ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60062025-10-03T21:18:11.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
60072025-10-03T21:18:11.090ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60082025-10-03T21:18:11.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60092025-10-03T21:18:11.091ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f7329e6-1bdc-431b-98a0-c3daa6b4ca79", "content-length": "682", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60102025-10-03T21:18:11.091ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60112025-10-03T21:18:11.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60122025-10-03T21:18:11.092ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60132025-10-03T21:18:11.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60142025-10-03T21:18:11.094ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60152025-10-03T21:18:11.095ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60162025-10-03T21:18:11.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60172025-10-03T21:18:11.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60182025-10-03T21:18:11.096ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
60192025-10-03T21:18:11.096ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.923057ms iteration = 35 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
60202025-10-03T21:18:11.098ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60212025-10-03T21:18:11.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60222025-10-03T21:18:11.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60232025-10-03T21:18:11.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60242025-10-03T21:18:11.104ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
60252025-10-03T21:18:11.104ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
60262025-10-03T21:18:11.107ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 915.000609ms iteration = 23 status = {}
60272025-10-03T21:18:11.135ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
60282025-10-03T21:18:11.136ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:55234
60292025-10-03T21:18:11.136ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:55234 req_id = 41eb9ffb-509d-445b-a772-6fea2d87586c roles = RoleSet { roles: {} } uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
60302025-10-03T21:18:11.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:55234 req_id = 41eb9ffb-509d-445b-a772-6fea2d87586c resource = Database result = Ok(()) uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
60312025-10-03T21:18:11.183ZINFOnexus: assigned collector to new producer collector_id = d03f63a5-05d6-4fcc-b79e-7d63884498be file = nexus/src/app/oximeter.rs:76 producer_id = a5fae325-6f04-4795-bc31-fbf69228d92b
60322025-10-03T21:18:11.183ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 46830 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:55234 req_id = 41eb9ffb-509d-445b-a772-6fea2d87586c response_code = 201 uri = /metrics/producers
60332025-10-03T21:18:11.183ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "41eb9ffb-509d-445b-a772-6fea2d87586c", "content-length": "41", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60342025-10-03T21:18:11.184ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
60352025-10-03T21:18:11.208ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(eadaeff6-9ae8-4553-bd1f-b9b3d921e683, 0), (31053175-4bf3-48bb-a958-177800eb5a29, 0), (dffb9ef4-bf37-4870-b52d-2f2d8ae79e87, 0), (df189a1a-0917-4749-bff1-6603c1d9275e, 0)] collection_id = eadaeff6-9ae8-4553-bd1f-b9b3d921e683
60362025-10-03T21:18:11.208ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60372025-10-03T21:18:11.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
60382025-10-03T21:18:11.225ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 400 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 1a7df50c-f487-4996-9e14-7a2c8c3a7772 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
60392025-10-03T21:18:11.368ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = eadaeff6-9ae8-4553-bd1f-b9b3d921e683 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
60402025-10-03T21:18:11.368ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60412025-10-03T21:18:11.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
60422025-10-03T21:18:11.454ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(dffb9ef4-bf37-4870-b52d-2f2d8ae79e87, 0), (df189a1a-0917-4749-bff1-6603c1d9275e, 0), (8e77a9b0-59eb-492d-aaf5-16d924b6dcdc, 0)]
60432025-10-03T21:18:11.455ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5136, tv_nsec: 942984383 } })
60442025-10-03T21:18:11.488ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("26cdd4d2-d0ac-43df-bf3e-350fd160efec.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41cf4311-4781-41d3-b8bc-181c344891b0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b55c423c-6cde-4185-879d-3a1c4cf01de4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b807e991-1a36-4877-8937-3efb3994dbc2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::3)) }], valid_until: Instant { tv_sec: 5136, tv_nsec: 975670562 } })
60452025-10-03T21:18:11.488ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
60462025-10-03T21:18:11.488ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
60472025-10-03T21:18:11.488ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
60482025-10-03T21:18:11.489ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b07fc9e-914c-46c2-a4e9-b99d93c023ab", "content-length": "476", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60492025-10-03T21:18:11.489ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
60502025-10-03T21:18:11.489ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "12af6d7f-e262-41ba-b000-f4e14e300cbf", "content-length": "734", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60512025-10-03T21:18:11.489ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
60522025-10-03T21:18:11.489ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13fe30ac-3f56-43fd-8d7b-c95a9c83535c", "content-length": "111", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60532025-10-03T21:18:11.489ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
60542025-10-03T21:18:11.490ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "824f4c07-ea5d-4c62-bdec-a2cd9baebd90", "content-length": "111", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60552025-10-03T21:18:11.490ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
60562025-10-03T21:18:11.490ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bf81adb-8ae0-42d2-a1d0-36eeb16c6446", "content-length": "163", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60572025-10-03T21:18:11.490ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
60582025-10-03T21:18:11.491ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a80a36ca-5dd0-4e93-9386-5f8d67b6c103", "content-length": "163", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60592025-10-03T21:18:11.491ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
60602025-10-03T21:18:11.491ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "39e1be78-af00-4f07-82af-a14fcfa25a80", "content-length": "166", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60612025-10-03T21:18:11.491ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
60622025-10-03T21:18:11.491ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9de9d59-89ba-441f-b30e-2af1c8815020", "content-length": "166", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60632025-10-03T21:18:11.491ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
60642025-10-03T21:18:11.491ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1512df16-4da3-4af9-a17d-c74cb724e6f2", "content-length": "702", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60652025-10-03T21:18:11.491ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
60662025-10-03T21:18:11.492ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad05ed6e-c211-4c8c-89b6-9f4399081681", "content-length": "727", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60672025-10-03T21:18:11.492ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
60682025-10-03T21:18:11.492ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31a5716b-0141-4dae-9cb4-200663e8e22e", "content-length": "729", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60692025-10-03T21:18:11.492ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
60702025-10-03T21:18:11.492ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a6d520b-1884-4fc4-801b-6b15bcb02c35", "content-length": "728", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60712025-10-03T21:18:11.492ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
60722025-10-03T21:18:11.492ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c61f5806-460b-4c3d-889a-2122e89b5f77", "content-length": "722", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60732025-10-03T21:18:11.492ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
60742025-10-03T21:18:11.493ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "64213357-95c7-44c2-9bda-2e00cf3590cc", "content-length": "10", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60752025-10-03T21:18:11.493ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
60762025-10-03T21:18:11.493ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4404dc2c-813d-4704-9f56-22cea709967b", "content-length": "144", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60772025-10-03T21:18:11.493ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
60782025-10-03T21:18:11.493ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b3dd810-103e-46b6-9bb1-23b9c8972f5e", "content-length": "147", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60792025-10-03T21:18:11.493ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
60802025-10-03T21:18:11.493ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a533cb8-2b74-4e23-a97e-fd5ed65ecbdd", "content-length": "109", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60812025-10-03T21:18:11.493ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
60822025-10-03T21:18:11.494ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "995aa211-dad3-4232-afb0-29783b8dcbe4", "content-length": "109", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60832025-10-03T21:18:11.494ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
60842025-10-03T21:18:11.494ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "feed5d38-58a2-4073-8d60-d0e8419c8ddf", "content-length": "163", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60852025-10-03T21:18:11.494ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
60862025-10-03T21:18:11.494ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2009916a-5666-401c-bedc-8cb4e1ef1541", "content-length": "163", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60872025-10-03T21:18:11.494ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
60882025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4014838b-90ff-4278-b842-a37946c6667a", "content-length": "166", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60892025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
60902025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc259220-6825-4714-9ec6-f8519189e5d2", "content-length": "166", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60912025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
60922025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0872ccc-1475-43e0-8e4b-71cf3672406f", "content-length": "702", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60932025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
60942025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db8dff50-90a2-42d1-ad46-ea7f1eb43ee1", "content-length": "727", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60952025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
60962025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "00ad9b55-6556-4f17-ab60-5a53ccdb3749", "content-length": "729", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60972025-10-03T21:18:11.495ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
60982025-10-03T21:18:11.496ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccedddb4-0d53-433f-b2e2-0724cd9715b0", "content-length": "728", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
60992025-10-03T21:18:11.496ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61002025-10-03T21:18:11.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61012025-10-03T21:18:11.498ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61022025-10-03T21:18:11.500ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61032025-10-03T21:18:11.500ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61042025-10-03T21:18:11.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61052025-10-03T21:18:11.502ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61062025-10-03T21:18:11.504ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61072025-10-03T21:18:11.504ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61082025-10-03T21:18:11.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61092025-10-03T21:18:11.540ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
61102025-10-03T21:18:11.540ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
61112025-10-03T21:18:11.651ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c561bcb8-7b5d-4de5-a6fb-fc908cf31f73", "content-length": "88632", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61122025-10-03T21:18:11.653ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61132025-10-03T21:18:11.653ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61142025-10-03T21:18:11.653ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
61152025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
61162025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
61172025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
61182025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
61192025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
61202025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6f85df7-61f5-4bcb-b005-a6af9bb120aa", "content-length": "64", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61212025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f83b7fc2-eab9-4c0d-b8a3-70ad84c6960e", "content-length": "64", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61222025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
61232025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
61242025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d92ed2c0-0815-42a1-9893-dda379e31923", "content-length": "64", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61252025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "80c62df7-27d6-47f9-9a30-23a7d6fe13dd", "content-length": "64", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61262025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
61272025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
61282025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7313e6e2-4b67-45e3-a5a0-4fdc19991819", "content-length": "64", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61292025-10-03T21:18:11.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
61302025-10-03T21:18:11.691ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecbc5110-7498-46a8-938a-058a540bdfcc", "content-length": "398199", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61312025-10-03T21:18:11.692ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "151daaf3-86b1-471c-98c1-2a573eb2e2ff", "content-length": "432321", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61322025-10-03T21:18:11.693ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "faae5805-2c57-4858-ac3f-2f60bf7f07e0", "content-length": "405629", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61332025-10-03T21:18:11.694ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ceb54874-1313-4cf2-a0c2-e44b8665aec9", "content-length": "405584", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61342025-10-03T21:18:11.694ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ee41a31-509c-4ac4-98fb-0c4a5835979e", "content-length": "408145", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61352025-10-03T21:18:11.697ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
61362025-10-03T21:18:11.697ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
61372025-10-03T21:18:11.697ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
61382025-10-03T21:18:11.697ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
61392025-10-03T21:18:11.697ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
61402025-10-03T21:18:11.730ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 (service)
61412025-10-03T21:18:11.730ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
61422025-10-03T21:18:11.733ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c926435-ede7-4cce-91b4-01bc3b0fdb14", "content-length": "123", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61432025-10-03T21:18:11.735ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
61442025-10-03T21:18:11.825ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
61452025-10-03T21:18:11.825ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
61462025-10-03T21:18:11.826ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "08ce79bc-0319-49d0-813c-02d8392b3f44", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61472025-10-03T21:18:11.826ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
61482025-10-03T21:18:11.828ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
61492025-10-03T21:18:11.828ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
61502025-10-03T21:18:11.828ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1201b1c2-5dec-4897-af52-b24cc185a52d", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61512025-10-03T21:18:11.828ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
61522025-10-03T21:18:11.830ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
61532025-10-03T21:18:11.830ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
61542025-10-03T21:18:11.830ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddf57eae-824f-4f9d-a5e4-26f8e42953e1", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:18:11 GMT"} })
61552025-10-03T21:18:11.830ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
61562025-10-03T21:18:11.832ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
61572025-10-03T21:18:11.832ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
61582025-10-03T21:18:11.835ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61592025-10-03T21:18:11.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61602025-10-03T21:18:11.838ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61612025-10-03T21:18:11.840ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
61622025-10-03T21:18:11.840ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61632025-10-03T21:18:11.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61642025-10-03T21:18:12.029ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = ce03add5-1108-49b7-ab5b-baeeab316a5f file = nexus/db-queries/src/db/datastore/inventory.rs:1614
61652025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = ce03add5-1108-49b7-ab5b-baeeab316a5f time_started = 2025-10-03 21:18:11.488823 UTC
61662025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 28 reason = Dependency
61672025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61682025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 37 reason = Dependency
61692025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 17 reason = Dependency
61702025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61712025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
61722025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 946.656445ms iteration = 14 status = {"collection_id":"ce03add5-1108-49b7-ab5b-baeeab316a5f","time_done":"2025-10-03 21:18:11.832784 UTC","time_started":"2025-10-03 21:18:11.488823 UTC"}
61732025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61742025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61752025-10-03T21:18:12.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
61762025-10-03T21:18:12.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61772025-10-03T21:18:12.032ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61782025-10-03T21:18:12.035ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
61792025-10-03T21:18:12.035ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61802025-10-03T21:18:12.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61812025-10-03T21:18:12.035ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
61822025-10-03T21:18:12.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
61832025-10-03T21:18:12.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
61842025-10-03T21:18:12.037ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61852025-10-03T21:18:12.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61862025-10-03T21:18:12.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
61872025-10-03T21:18:12.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
61882025-10-03T21:18:12.038ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61892025-10-03T21:18:12.040ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
61902025-10-03T21:18:12.040ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61912025-10-03T21:18:12.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61922025-10-03T21:18:12.055ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 25.951143ms iteration = 37 status = {"physical_disks_added":0}
61932025-10-03T21:18:12.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
61942025-10-03T21:18:12.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
61952025-10-03T21:18:12.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61962025-10-03T21:18:12.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61972025-10-03T21:18:12.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61982025-10-03T21:18:12.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61992025-10-03T21:18:12.081ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62002025-10-03T21:18:12.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62012025-10-03T21:18:12.081ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62022025-10-03T21:18:12.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62032025-10-03T21:18:12.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62042025-10-03T21:18:12.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62052025-10-03T21:18:12.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62062025-10-03T21:18:12.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62072025-10-03T21:18:12.085ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62082025-10-03T21:18:12.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62092025-10-03T21:18:12.085ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62102025-10-03T21:18:12.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62112025-10-03T21:18:12.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62122025-10-03T21:18:12.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62132025-10-03T21:18:12.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62142025-10-03T21:18:12.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62152025-10-03T21:18:12.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62162025-10-03T21:18:12.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62172025-10-03T21:18:12.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62182025-10-03T21:18:12.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62192025-10-03T21:18:12.090ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62202025-10-03T21:18:12.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62212025-10-03T21:18:12.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62222025-10-03T21:18:12.091ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62232025-10-03T21:18:12.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62242025-10-03T21:18:12.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62252025-10-03T21:18:12.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62262025-10-03T21:18:12.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62272025-10-03T21:18:12.094ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62282025-10-03T21:18:12.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62292025-10-03T21:18:12.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62302025-10-03T21:18:12.095ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62312025-10-03T21:18:12.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62322025-10-03T21:18:12.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62332025-10-03T21:18:12.097ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
62342025-10-03T21:18:12.097ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 67.758815ms iteration = 17 status = {"blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","inventory_collection_id":"ce03add5-1108-49b7-ab5b-baeeab316a5f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
62352025-10-03T21:18:12.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62362025-10-03T21:18:12.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62372025-10-03T21:18:12.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62382025-10-03T21:18:12.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62392025-10-03T21:18:12.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62402025-10-03T21:18:12.106ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62412025-10-03T21:18:12.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62422025-10-03T21:18:12.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62432025-10-03T21:18:12.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62442025-10-03T21:18:12.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62452025-10-03T21:18:12.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62462025-10-03T21:18:12.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
62472025-10-03T21:18:12.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62482025-10-03T21:18:12.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62492025-10-03T21:18:12.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62502025-10-03T21:18:12.116ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62512025-10-03T21:18:12.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62522025-10-03T21:18:12.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62532025-10-03T21:18:12.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62542025-10-03T21:18:12.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62552025-10-03T21:18:12.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62562025-10-03T21:18:12.123ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
62572025-10-03T21:18:12.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62582025-10-03T21:18:12.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62592025-10-03T21:18:12.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62602025-10-03T21:18:12.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62612025-10-03T21:18:12.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62622025-10-03T21:18:12.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62632025-10-03T21:18:12.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62642025-10-03T21:18:12.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62652025-10-03T21:18:12.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62662025-10-03T21:18:12.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62672025-10-03T21:18:12.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62682025-10-03T21:18:12.134ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62692025-10-03T21:18:12.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62702025-10-03T21:18:12.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62712025-10-03T21:18:12.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62722025-10-03T21:18:12.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62732025-10-03T21:18:12.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62742025-10-03T21:18:12.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62752025-10-03T21:18:12.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62762025-10-03T21:18:12.142ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62772025-10-03T21:18:12.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62782025-10-03T21:18:12.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62792025-10-03T21:18:12.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62802025-10-03T21:18:12.145ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62812025-10-03T21:18:12.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62822025-10-03T21:18:12.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62832025-10-03T21:18:12.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62842025-10-03T21:18:12.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62852025-10-03T21:18:12.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62862025-10-03T21:18:12.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62872025-10-03T21:18:12.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62882025-10-03T21:18:12.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
62892025-10-03T21:18:12.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62902025-10-03T21:18:12.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62912025-10-03T21:18:12.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62922025-10-03T21:18:12.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62932025-10-03T21:18:12.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62942025-10-03T21:18:12.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
62952025-10-03T21:18:12.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62962025-10-03T21:18:12.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62972025-10-03T21:18:12.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62982025-10-03T21:18:12.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62992025-10-03T21:18:12.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63002025-10-03T21:18:12.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
63012025-10-03T21:18:12.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63022025-10-03T21:18:12.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63032025-10-03T21:18:12.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63042025-10-03T21:18:12.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63052025-10-03T21:18:12.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63062025-10-03T21:18:12.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
63072025-10-03T21:18:12.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63082025-10-03T21:18:12.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63092025-10-03T21:18:12.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63102025-10-03T21:18:12.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63112025-10-03T21:18:12.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63122025-10-03T21:18:12.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
63132025-10-03T21:18:12.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63142025-10-03T21:18:12.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63152025-10-03T21:18:12.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63162025-10-03T21:18:12.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63172025-10-03T21:18:12.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63182025-10-03T21:18:12.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
63192025-10-03T21:18:12.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63202025-10-03T21:18:12.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63212025-10-03T21:18:12.214ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
63222025-10-03T21:18:12.214ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
63232025-10-03T21:18:12.214ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 7bf9be3c-e446-4e7d-9386-9c0db207a993 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
63242025-10-03T21:18:12.214ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
63252025-10-03T21:18:12.214ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 184.665115ms iteration = 28 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
63262025-10-03T21:18:15.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 134 reason = Timeout
63272025-10-03T21:18:15.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
63282025-10-03T21:18:15.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
63292025-10-03T21:18:15.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63302025-10-03T21:18:15.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
63312025-10-03T21:18:15.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
63322025-10-03T21:18:15.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
63332025-10-03T21:18:15.198ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
63342025-10-03T21:18:15.198ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.505873ms iteration = 134 status = {"config_updated":false}
63352025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
63362025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
63372025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63382025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63392025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 135 reason = Timeout
63402025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
63412025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63422025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
63432025-10-03T21:18:20.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
63442025-10-03T21:18:20.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63452025-10-03T21:18:20.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63462025-10-03T21:18:20.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
63472025-10-03T21:18:20.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
63482025-10-03T21:18:20.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
63492025-10-03T21:18:20.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63502025-10-03T21:18:20.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
63512025-10-03T21:18:20.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
63522025-10-03T21:18:20.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
63532025-10-03T21:18:20.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.626896ms iteration = 135 status = {"config_updated":false}
63542025-10-03T21:18:20.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63552025-10-03T21:18:20.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
63562025-10-03T21:18:20.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63572025-10-03T21:18:20.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
63582025-10-03T21:18:20.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63592025-10-03T21:18:20.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
63602025-10-03T21:18:20.219ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
63612025-10-03T21:18:20.219ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.004004ms iteration = 68 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
63622025-10-03T21:18:20.227ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63632025-10-03T21:18:20.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63642025-10-03T21:18:20.229ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.31159ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
63652025-10-03T21:18:21.218ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:44624
63662025-10-03T21:18:21.218ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 35162e3b-5e99-45b8-ad94-0b4b7a347136 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
63672025-10-03T21:18:21.219ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 35162e3b-5e99-45b8-ad94-0b4b7a347136 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
63682025-10-03T21:18:21.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3379 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 35162e3b-5e99-45b8-ad94-0b4b7a347136 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending
63692025-10-03T21:18:21.222ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 35a530f1-2539-47c9-b241-0982c735b367 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
63702025-10-03T21:18:21.222ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 35a530f1-2539-47c9-b241-0982c735b367 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
63712025-10-03T21:18:21.224ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 35a530f1-2539-47c9-b241-0982c735b367 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D
63722025-10-03T21:18:21.225ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 412 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 780f9719-f4bd-44c5-8f12-f517d135e7c1 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
63732025-10-03T21:18:25.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 136 reason = Timeout
63742025-10-03T21:18:25.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
63752025-10-03T21:18:25.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
63762025-10-03T21:18:25.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63772025-10-03T21:18:25.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
63782025-10-03T21:18:25.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
63792025-10-03T21:18:25.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
63802025-10-03T21:18:25.199ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
63812025-10-03T21:18:25.199ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.309012ms iteration = 136 status = {"config_updated":false}
63822025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
63832025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
63842025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63852025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63862025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 137 reason = Timeout
63872025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
63882025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
63892025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63902025-10-03T21:18:30.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
63912025-10-03T21:18:30.195ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63922025-10-03T21:18:30.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63932025-10-03T21:18:30.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63942025-10-03T21:18:30.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63952025-10-03T21:18:30.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.165727ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
63962025-10-03T21:18:30.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
63972025-10-03T21:18:30.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63982025-10-03T21:18:30.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
63992025-10-03T21:18:30.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
64002025-10-03T21:18:30.199ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64012025-10-03T21:18:30.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64022025-10-03T21:18:30.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
64032025-10-03T21:18:30.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.335423ms iteration = 137 status = {"config_updated":false}
64042025-10-03T21:18:30.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64052025-10-03T21:18:30.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64062025-10-03T21:18:30.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64072025-10-03T21:18:30.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
64082025-10-03T21:18:30.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64092025-10-03T21:18:30.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64102025-10-03T21:18:30.220ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
64112025-10-03T21:18:30.220ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.373349ms iteration = 69 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
64122025-10-03T21:18:31.227ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 733 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = c6af2c1f-a4fb-4e78-883c-bd58be8876ef response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
64132025-10-03T21:18:35.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 138 reason = Timeout
64142025-10-03T21:18:35.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64152025-10-03T21:18:35.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64162025-10-03T21:18:35.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64172025-10-03T21:18:35.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
64182025-10-03T21:18:35.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64192025-10-03T21:18:35.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64202025-10-03T21:18:35.199ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
64212025-10-03T21:18:35.199ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.227361ms iteration = 138 status = {"config_updated":false}
64222025-10-03T21:18:40.191ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
64232025-10-03T21:18:40.192ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
64242025-10-03T21:18:40.192ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
64252025-10-03T21:18:40.192ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
64262025-10-03T21:18:40.192ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
64272025-10-03T21:18:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
64282025-10-03T21:18:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
64292025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 36 reason = Timeout
64302025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
64312025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
64322025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64332025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64342025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
64352025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
64362025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
64372025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
64382025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
64392025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64402025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
64412025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
64422025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64432025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
64442025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64452025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
64462025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64472025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
64482025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64492025-10-03T21:18:40.193ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
64502025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64512025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
64522025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
64532025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 139 reason = Timeout
64542025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64552025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
64562025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
64572025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 38 reason = Timeout
64582025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64592025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
64602025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
64612025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
64622025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
64632025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
64642025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
64652025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
64662025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.46\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"sps":[]}
64672025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64682025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
64692025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64702025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
64712025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
64722025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
64732025-10-03T21:18:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
64742025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64752025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64762025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64772025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
64782025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
64792025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64802025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64812025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64822025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
64832025-10-03T21:18:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
64842025-10-03T21:18:40.196ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64852025-10-03T21:18:40.196ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64862025-10-03T21:18:40.196ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5165, tv_nsec: 683698178 } })
64872025-10-03T21:18:40.196ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5165, tv_nsec: 683906570 } })
64882025-10-03T21:18:40.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64892025-10-03T21:18:40.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64902025-10-03T21:18:40.197ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
64912025-10-03T21:18:40.197ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64922025-10-03T21:18:40.197ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64932025-10-03T21:18:40.197ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
64942025-10-03T21:18:40.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64952025-10-03T21:18:40.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.266388ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
64962025-10-03T21:18:40.197ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64972025-10-03T21:18:40.197ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64982025-10-03T21:18:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
64992025-10-03T21:18:40.202ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
65002025-10-03T21:18:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
65012025-10-03T21:18:40.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65022025-10-03T21:18:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
65032025-10-03T21:18:40.202ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.911453ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
65042025-10-03T21:18:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65052025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
65062025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65072025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
65082025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65092025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
65102025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65112025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65122025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
65132025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65142025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
65152025-10-03T21:18:40.203ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
65162025-10-03T21:18:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
65172025-10-03T21:18:40.204ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
65182025-10-03T21:18:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
65192025-10-03T21:18:40.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65202025-10-03T21:18:40.206ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
65212025-10-03T21:18:40.206ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
65222025-10-03T21:18:40.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65232025-10-03T21:18:40.206ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
65242025-10-03T21:18:40.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
65252025-10-03T21:18:40.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
65262025-10-03T21:18:40.207ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.335832ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65272025-10-03T21:18:40.207ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 14.356872ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
65282025-10-03T21:18:40.208ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65292025-10-03T21:18:40.208ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65302025-10-03T21:18:40.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65312025-10-03T21:18:40.209ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
65322025-10-03T21:18:40.209ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 15.655367ms iteration = 139 status = {"config_updated":false}
65332025-10-03T21:18:40.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
65342025-10-03T21:18:40.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65352025-10-03T21:18:40.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65362025-10-03T21:18:40.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65372025-10-03T21:18:40.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
65382025-10-03T21:18:40.211ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5165, tv_nsec: 698543836 } })
65392025-10-03T21:18:40.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65402025-10-03T21:18:40.211ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
65412025-10-03T21:18:40.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65422025-10-03T21:18:40.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65432025-10-03T21:18:40.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
65442025-10-03T21:18:40.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
65452025-10-03T21:18:40.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
65462025-10-03T21:18:40.215ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65472025-10-03T21:18:40.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
65482025-10-03T21:18:40.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
65492025-10-03T21:18:40.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65502025-10-03T21:18:40.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
65512025-10-03T21:18:40.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
65522025-10-03T21:18:40.218ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65532025-10-03T21:18:40.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65542025-10-03T21:18:40.220ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 27.111186ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
65552025-10-03T21:18:40.226ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 33.513868ms iteration = 38 status = {"physical_disks_added":0}
65562025-10-03T21:18:40.252ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65572025-10-03T21:18:40.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
65582025-10-03T21:18:40.263ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
65592025-10-03T21:18:40.264ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 70.682198ms iteration = 70 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
65602025-10-03T21:18:40.265ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65612025-10-03T21:18:40.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65622025-10-03T21:18:40.269ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65632025-10-03T21:18:40.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65642025-10-03T21:18:40.271ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 78.748499ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
65652025-10-03T21:18:40.272ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65662025-10-03T21:18:40.275ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
65672025-10-03T21:18:40.275ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65682025-10-03T21:18:40.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65692025-10-03T21:18:40.277ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
65702025-10-03T21:18:40.277ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
65712025-10-03T21:18:40.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
65722025-10-03T21:18:40.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65732025-10-03T21:18:40.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
65742025-10-03T21:18:40.280ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
65752025-10-03T21:18:40.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65762025-10-03T21:18:40.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65772025-10-03T21:18:40.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 91.035758ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
65782025-10-03T21:18:40.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 94.582208ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
65792025-10-03T21:18:40.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65802025-10-03T21:18:40.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65812025-10-03T21:18:40.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65822025-10-03T21:18:40.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65832025-10-03T21:18:40.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65842025-10-03T21:18:40.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
65852025-10-03T21:18:40.300ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65862025-10-03T21:18:40.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65872025-10-03T21:18:40.305ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 112.597161ms iteration = 24 status = 0
65882025-10-03T21:18:40.307ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
65892025-10-03T21:18:40.307ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 114.571763ms iteration = 36 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
65902025-10-03T21:18:40.335ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65912025-10-03T21:18:40.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
65922025-10-03T21:18:40.338ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65932025-10-03T21:18:40.340ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
65942025-10-03T21:18:40.340ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65952025-10-03T21:18:40.341ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2522229-0f7a-44a4-a6fa-ec7a07f4c570", "content-length": "26", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
65962025-10-03T21:18:40.341ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65972025-10-03T21:18:40.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
65982025-10-03T21:18:40.341ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65992025-10-03T21:18:40.342ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
66002025-10-03T21:18:40.342ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66012025-10-03T21:18:40.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
66022025-10-03T21:18:40.342ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbddd571-9d6c-4f10-b9ed-e7fa91025bec", "content-length": "26", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66032025-10-03T21:18:40.342ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66042025-10-03T21:18:40.344ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66052025-10-03T21:18:40.346ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
66062025-10-03T21:18:40.346ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
66072025-10-03T21:18:40.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66082025-10-03T21:18:40.347ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66092025-10-03T21:18:40.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
66102025-10-03T21:18:40.348ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
66112025-10-03T21:18:40.348ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
66122025-10-03T21:18:40.348ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/30a5811c-f86d-4d5d-95bf-4c3c64e1280d/state
66132025-10-03T21:18:40.348ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 155.082551ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
66142025-10-03T21:18:40.348ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/30a5811c-f86d-4d5d-95bf-4c3c64e1280d/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6fb6873-5e0e-469d-a305-120c4c8f8782", "content-length": "130", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66152025-10-03T21:18:40.351ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = ec08e8e8-f3ad-4fff-8212-9390026f6316 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-10-03T21:18:37.200587122Z }, migration_in: None, migration_out: None } vmm_id = 30a5811c-f86d-4d5d-95bf-4c3c64e1280d
66162025-10-03T21:18:40.351ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2150 instance_id = ec08e8e8-f3ad-4fff-8212-9390026f6316 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 30a5811c-f86d-4d5d-95bf-4c3c64e1280d sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 vmm_id = 30a5811c-f86d-4d5d-95bf-4c3c64e1280d vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-10-03T21:18:37.200587122Z }
66172025-10-03T21:18:40.351ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ec08e8e8-f3ad-4fff-8212-9390026f6316 roles = RoleSet { roles: {} } sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 vmm_id = 30a5811c-f86d-4d5d-95bf-4c3c64e1280d
66182025-10-03T21:18:40.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher instance_id = ec08e8e8-f3ad-4fff-8212-9390026f6316 resource = Database result = Ok(()) sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 vmm_id = 30a5811c-f86d-4d5d-95bf-4c3c64e1280d
66192025-10-03T21:18:40.358ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
66202025-10-03T21:18:40.358ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66212025-10-03T21:18:40.359ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a671274-aa02-49b0-bfbe-36d3ab8ffedc", "content-length": "26", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66222025-10-03T21:18:40.359ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66232025-10-03T21:18:40.362ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
66242025-10-03T21:18:40.363ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
66252025-10-03T21:18:40.365ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 172.058773ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
66262025-10-03T21:18:40.444ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
66272025-10-03T21:18:40.444ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66282025-10-03T21:18:40.445ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6146c9c5-f21c-4700-a8e2-2a9bf519f65f", "content-length": "583", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66292025-10-03T21:18:40.445ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66302025-10-03T21:18:40.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66312025-10-03T21:18:40.468ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {5fdf0a88-d6a5-4fbb-af21-df95f33f7621: {10.151.1.176: {7b942a07-703c-4955-8f5c-9c52fc4fcfac}, 10.151.1.175: {7b942a07-703c-4955-8f5c-9c52fc4fcfac}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
66322025-10-03T21:18:40.468ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
66332025-10-03T21:18:40.468ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "913e7137-ddac-4bf9-a4f2-2420572bec80", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66342025-10-03T21:18:40.468ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66352025-10-03T21:18:40.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66362025-10-03T21:18:40.472ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66372025-10-03T21:18:40.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66382025-10-03T21:18:40.476ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66392025-10-03T21:18:40.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66402025-10-03T21:18:40.480ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66412025-10-03T21:18:40.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66422025-10-03T21:18:40.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66432025-10-03T21:18:40.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66442025-10-03T21:18:40.487ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66452025-10-03T21:18:40.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66462025-10-03T21:18:40.491ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
66472025-10-03T21:18:40.491ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
66482025-10-03T21:18:40.491ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66492025-10-03T21:18:40.491ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "470b493d-4b69-45b1-8396-1f0ed1eddbcb", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66502025-10-03T21:18:40.493ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 299.912645ms iteration = 24 status = {}
66512025-10-03T21:18:40.507ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
66522025-10-03T21:18:40.507ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "231341a0-901c-4cce-afdd-719a8b015d56", "content-length": "810", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66532025-10-03T21:18:40.507ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(6139818) }} sled = buskin
66542025-10-03T21:18:40.507ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
66552025-10-03T21:18:40.507ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
66562025-10-03T21:18:40.509ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 316.104888ms iteration = 24 status = {}
66572025-10-03T21:18:40.669ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 477.693702ms iteration = 24 status = {}
66582025-10-03T21:18:40.673ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
66592025-10-03T21:18:40.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
66602025-10-03T21:18:40.676ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 484.775392ms iteration = 24 status = null
66612025-10-03T21:18:40.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66622025-10-03T21:18:40.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66632025-10-03T21:18:40.735ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
66642025-10-03T21:18:40.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66652025-10-03T21:18:40.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66662025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
66672025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
66682025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
66692025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
66702025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66712025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66722025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66732025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
66742025-10-03T21:18:40.748ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66752025-10-03T21:18:40.749ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
66762025-10-03T21:18:40.749ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
66772025-10-03T21:18:40.749ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66782025-10-03T21:18:40.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66792025-10-03T21:18:40.752ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
66802025-10-03T21:18:40.752ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
66812025-10-03T21:18:40.914ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4a426157-b398-4865-bd33-cda97fe011a8", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66822025-10-03T21:18:40.914ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
66832025-10-03T21:18:40.914ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
66842025-10-03T21:18:40.914ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a109d9af-55bc-4b6d-9320-809cb1b4ac6b", "content-length": "682", "date": "Fri, 03 Oct 2025 21:18:40 GMT"} })
66852025-10-03T21:18:40.914ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66862025-10-03T21:18:40.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66872025-10-03T21:18:40.919ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66882025-10-03T21:18:40.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66892025-10-03T21:18:40.923ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66902025-10-03T21:18:40.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66912025-10-03T21:18:40.925ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66922025-10-03T21:18:40.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66932025-10-03T21:18:40.932ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 739.681819ms iteration = 24 status = {}
66942025-10-03T21:18:41.233ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 7786 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 0529f666-6854-4266-9132-54b31bc544a8 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
66952025-10-03T21:18:45.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 140 reason = Timeout
66962025-10-03T21:18:45.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
66972025-10-03T21:18:45.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
66982025-10-03T21:18:45.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66992025-10-03T21:18:45.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
67002025-10-03T21:18:45.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67012025-10-03T21:18:45.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67022025-10-03T21:18:45.199ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
67032025-10-03T21:18:45.199ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.951649ms iteration = 140 status = {"config_updated":false}
67042025-10-03T21:18:50.192ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
67052025-10-03T21:18:50.192ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67062025-10-03T21:18:50.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
67072025-10-03T21:18:50.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 141 reason = Timeout
67082025-10-03T21:18:50.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67092025-10-03T21:18:50.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67102025-10-03T21:18:50.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67112025-10-03T21:18:50.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67122025-10-03T21:18:50.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
67132025-10-03T21:18:50.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67142025-10-03T21:18:50.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67152025-10-03T21:18:50.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
67162025-10-03T21:18:50.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67172025-10-03T21:18:50.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67182025-10-03T21:18:50.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67192025-10-03T21:18:50.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67202025-10-03T21:18:50.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
67212025-10-03T21:18:50.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
67222025-10-03T21:18:50.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.312953ms iteration = 141 status = {"config_updated":false}
67232025-10-03T21:18:50.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67242025-10-03T21:18:50.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
67252025-10-03T21:18:50.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67262025-10-03T21:18:50.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
67272025-10-03T21:18:50.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67282025-10-03T21:18:50.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
67292025-10-03T21:18:50.225ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
67302025-10-03T21:18:50.225ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.041143ms iteration = 71 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
67312025-10-03T21:18:50.236ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67322025-10-03T21:18:50.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67332025-10-03T21:18:50.238ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 46.04771ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67342025-10-03T21:18:51.219ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 52934ca7-9bcd-41cf-826c-80af17a412cc roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
67352025-10-03T21:18:51.219ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 52934ca7-9bcd-41cf-826c-80af17a412cc resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
67362025-10-03T21:18:51.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3105 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = 52934ca7-9bcd-41cf-826c-80af17a412cc response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending
67372025-10-03T21:18:51.222ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = ec061ef2-4fe1-449d-994c-84a7e150f793 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
67382025-10-03T21:18:51.222ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = ec061ef2-4fe1-449d-994c-84a7e150f793 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
67392025-10-03T21:18:51.224ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2409 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44624 req_id = ec061ef2-4fe1-449d-994c-84a7e150f793 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D
67402025-10-03T21:18:51.226ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 419 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 50097289-557d-4ae8-ae67-31849216e960 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
67412025-10-03T21:18:55.192ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 142 reason = Timeout
67422025-10-03T21:18:55.192ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67432025-10-03T21:18:55.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67442025-10-03T21:18:55.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67452025-10-03T21:18:55.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
67462025-10-03T21:18:55.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67472025-10-03T21:18:55.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67482025-10-03T21:18:55.199ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
67492025-10-03T21:18:55.199ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.933378ms iteration = 142 status = {"config_updated":false}
67502025-10-03T21:19:00.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
67512025-10-03T21:19:00.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67522025-10-03T21:19:00.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
67532025-10-03T21:19:00.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67542025-10-03T21:19:00.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 143 reason = Timeout
67552025-10-03T21:19:00.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67562025-10-03T21:19:00.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
67572025-10-03T21:19:00.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67582025-10-03T21:19:00.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67592025-10-03T21:19:00.194ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67602025-10-03T21:19:00.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67612025-10-03T21:19:00.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67622025-10-03T21:19:00.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67632025-10-03T21:19:00.196ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.198806ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67642025-10-03T21:19:00.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67652025-10-03T21:19:00.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67662025-10-03T21:19:00.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
67672025-10-03T21:19:00.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67682025-10-03T21:19:00.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
67692025-10-03T21:19:00.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67702025-10-03T21:19:00.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
67712025-10-03T21:19:00.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.133781ms iteration = 143 status = {"config_updated":false}
67722025-10-03T21:19:00.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67732025-10-03T21:19:00.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
67742025-10-03T21:19:00.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67752025-10-03T21:19:00.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
67762025-10-03T21:19:00.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67772025-10-03T21:19:00.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
67782025-10-03T21:19:00.220ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
67792025-10-03T21:19:00.220ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.535251ms iteration = 72 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
67802025-10-03T21:19:01.226ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 686 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = ca845e5e-a0d6-4790-ac78-54fc1d1e501d response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
67812025-10-03T21:19:05.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 144 reason = Timeout
67822025-10-03T21:19:05.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67832025-10-03T21:19:05.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67842025-10-03T21:19:05.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67852025-10-03T21:19:05.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
67862025-10-03T21:19:05.197ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
67872025-10-03T21:19:05.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
67882025-10-03T21:19:05.199ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
67892025-10-03T21:19:05.199ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.754266ms iteration = 144 status = {"config_updated":false}
67902025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
67912025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
67922025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
67932025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
67942025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
67952025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
67962025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
67972025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
67982025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
67992025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
68002025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
68012025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
68022025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
68032025-10-03T21:19:10.192ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
68042025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
68052025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
68062025-10-03T21:19:10.192ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = ExternalNetworking step id = Ensure
68072025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
68082025-10-03T21:19:10.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68092025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
68102025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
68112025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
68122025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
68132025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
68142025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
68152025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
68162025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
68172025-10-03T21:19:10.193ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-10-03 21:09:10.192978751 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
68182025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
68192025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
68202025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 29 reason = Timeout
68212025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68222025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
68232025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
68242025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68252025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
68262025-10-03T21:19:10.193ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
68272025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
68282025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
68292025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68302025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
68312025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68322025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68332025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68342025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
68352025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68362025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 145 reason = Timeout
68372025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
68382025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
68392025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
68402025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
68412025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
68422025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
68432025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
68442025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
68452025-10-03T21:19:10.193ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
68462025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
68472025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
68482025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
68492025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
68502025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
68512025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
68522025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
68532025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
68542025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
68552025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 39 reason = Timeout
68562025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68572025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
68582025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
68592025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 37 reason = Timeout
68602025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
68612025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
68622025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
68632025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68642025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
68652025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
68662025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
68672025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
68682025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
68692025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.66\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"sps":[]}
68702025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68712025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
68722025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
68732025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
68742025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
68752025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
68762025-10-03T21:19:10.193ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("486849a9-1733-4aa9-9b66-43ace2da0eea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 5195, tv_nsec: 680317740 } })
68772025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68782025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68792025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68802025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
68812025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
68822025-10-03T21:19:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68832025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
68842025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
68852025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
68862025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
68872025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
68882025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68892025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
68902025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
68912025-10-03T21:19:10.194ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2efcfb05-0c3d-45aa-b120-c334f3706be4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a7c1656-d513-4cc3-b2b0-8e31ad2b5545.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 5195, tv_nsec: 680584593 } })
68922025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
68932025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
68942025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
68952025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
68962025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
68972025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
68982025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
68992025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
69002025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
69012025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
69022025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
69032025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.576548ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
69042025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69052025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
69062025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.548487ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
69072025-10-03T21:19:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
69082025-10-03T21:19:10.195ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69092025-10-03T21:19:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69102025-10-03T21:19:10.195ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69112025-10-03T21:19:10.195ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69122025-10-03T21:19:10.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69132025-10-03T21:19:10.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69142025-10-03T21:19:10.196ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69152025-10-03T21:19:10.196ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69162025-10-03T21:19:10.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69172025-10-03T21:19:10.196ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69182025-10-03T21:19:10.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69192025-10-03T21:19:10.197ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69202025-10-03T21:19:10.197ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.075937ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
69212025-10-03T21:19:10.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.460191ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
69222025-10-03T21:19:10.198ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69232025-10-03T21:19:10.198ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-10-03 21:09:10.192978751 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
69242025-10-03T21:19:10.198ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.972967ms iteration = 13 status = {"expiration":"2025-10-03T21:09:10.192978751Z","pruned":[]}
69252025-10-03T21:19:10.199ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 6.035258ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
69262025-10-03T21:19:10.199ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69272025-10-03T21:19:10.199ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
69282025-10-03T21:19:10.199ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
69292025-10-03T21:19:10.199ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
69302025-10-03T21:19:10.199ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 6.436373ms iteration = 13 status = {"by_rx":{},"error":null}
69312025-10-03T21:19:10.200ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69322025-10-03T21:19:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
69332025-10-03T21:19:10.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69342025-10-03T21:19:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69352025-10-03T21:19:10.201ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.280714ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
69362025-10-03T21:19:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
69372025-10-03T21:19:10.201ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69382025-10-03T21:19:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
69392025-10-03T21:19:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
69402025-10-03T21:19:10.201ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69412025-10-03T21:19:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69422025-10-03T21:19:10.202ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69432025-10-03T21:19:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69442025-10-03T21:19:10.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69452025-10-03T21:19:10.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69462025-10-03T21:19:10.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69472025-10-03T21:19:10.204ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5195, tv_nsec: 690964320 } })
69482025-10-03T21:19:10.204ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69492025-10-03T21:19:10.204ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
69502025-10-03T21:19:10.204ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5195, tv_nsec: 691593387 } })
69512025-10-03T21:19:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
69522025-10-03T21:19:10.205ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
69532025-10-03T21:19:10.205ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
69542025-10-03T21:19:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
69552025-10-03T21:19:10.206ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69562025-10-03T21:19:10.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
69572025-10-03T21:19:10.207ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69582025-10-03T21:19:10.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69592025-10-03T21:19:10.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69602025-10-03T21:19:10.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69612025-10-03T21:19:10.209ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
69622025-10-03T21:19:10.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69632025-10-03T21:19:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
69642025-10-03T21:19:10.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69652025-10-03T21:19:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69662025-10-03T21:19:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69672025-10-03T21:19:10.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
69682025-10-03T21:19:10.212ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
69692025-10-03T21:19:10.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
69702025-10-03T21:19:10.212ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69712025-10-03T21:19:10.215ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69722025-10-03T21:19:10.215ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
69732025-10-03T21:19:10.215ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
69742025-10-03T21:19:10.215ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
69752025-10-03T21:19:10.215ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69762025-10-03T21:19:10.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69772025-10-03T21:19:10.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
69782025-10-03T21:19:10.216ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69792025-10-03T21:19:10.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69802025-10-03T21:19:10.217ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69812025-10-03T21:19:10.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
69822025-10-03T21:19:10.220ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69832025-10-03T21:19:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69842025-10-03T21:19:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
69852025-10-03T21:19:10.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69862025-10-03T21:19:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69872025-10-03T21:19:10.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69882025-10-03T21:19:10.223ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
69892025-10-03T21:19:10.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
69902025-10-03T21:19:10.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
69912025-10-03T21:19:10.224ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69922025-10-03T21:19:10.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69932025-10-03T21:19:10.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69942025-10-03T21:19:10.225ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69952025-10-03T21:19:10.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69962025-10-03T21:19:10.228ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
69972025-10-03T21:19:10.228ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69982025-10-03T21:19:10.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69992025-10-03T21:19:10.228ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5195, tv_nsec: 715433966 } })
70002025-10-03T21:19:10.229ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
70012025-10-03T21:19:10.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
70022025-10-03T21:19:10.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
70032025-10-03T21:19:10.241ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70042025-10-03T21:19:10.243ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70052025-10-03T21:19:10.248ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
70062025-10-03T21:19:10.248ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70072025-10-03T21:19:10.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70082025-10-03T21:19:10.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
70092025-10-03T21:19:10.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70102025-10-03T21:19:10.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
70112025-10-03T21:19:10.252ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 59.549052ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
70122025-10-03T21:19:10.254ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70132025-10-03T21:19:10.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
70142025-10-03T21:19:10.260ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70152025-10-03T21:19:10.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70162025-10-03T21:19:10.262ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70172025-10-03T21:19:10.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70182025-10-03T21:19:10.262ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 69.770208ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"195.0.0","globs_reprocessed":{},"no_receivers":[]}
70192025-10-03T21:19:10.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70202025-10-03T21:19:10.269ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 76.367191ms iteration = 6 status = {"nsuccess":5}
70212025-10-03T21:19:10.270ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70222025-10-03T21:19:10.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70232025-10-03T21:19:10.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
70242025-10-03T21:19:10.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
70252025-10-03T21:19:10.274ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
70262025-10-03T21:19:10.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
70272025-10-03T21:19:10.275ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
70282025-10-03T21:19:10.275ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
70292025-10-03T21:19:10.275ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
70302025-10-03T21:19:10.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70312025-10-03T21:19:10.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70322025-10-03T21:19:10.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
70332025-10-03T21:19:10.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70342025-10-03T21:19:10.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
70352025-10-03T21:19:10.277ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
70362025-10-03T21:19:10.279ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
70372025-10-03T21:19:10.281ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
70382025-10-03T21:19:10.282ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
70392025-10-03T21:19:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
70402025-10-03T21:19:10.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70412025-10-03T21:19:10.289ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
70422025-10-03T21:19:10.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
70432025-10-03T21:19:10.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70442025-10-03T21:19:10.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70452025-10-03T21:19:10.296ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
70462025-10-03T21:19:10.298ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
70472025-10-03T21:19:10.301ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
70482025-10-03T21:19:10.303ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
70492025-10-03T21:19:10.306ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
70502025-10-03T21:19:10.308ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70512025-10-03T21:19:10.311ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
70522025-10-03T21:19:10.312ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
70532025-10-03T21:19:10.312ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
70542025-10-03T21:19:10.312ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 118.956263ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
70552025-10-03T21:19:10.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
70562025-10-03T21:19:10.312ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
70572025-10-03T21:19:10.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
70582025-10-03T21:19:10.315ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70592025-10-03T21:19:10.315ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
70602025-10-03T21:19:10.316ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = SledList step id = Fetch
70612025-10-03T21:19:10.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70622025-10-03T21:19:10.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70632025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
70642025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
70652025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
70662025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70672025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
70682025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
70692025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
70702025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
70712025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-10-03 21:07:12.445842 UTC dns_group = internal generation = 1
70722025-10-03T21:19:10.318ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 125.861901ms iteration = 13 status = {"generation":1}
70732025-10-03T21:19:10.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
70742025-10-03T21:19:10.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
70752025-10-03T21:19:10.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70762025-10-03T21:19:10.325ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
70772025-10-03T21:19:10.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70782025-10-03T21:19:10.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70792025-10-03T21:19:10.330ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
70802025-10-03T21:19:10.330ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
70812025-10-03T21:19:10.330ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
70822025-10-03T21:19:10.330ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70832025-10-03T21:19:10.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
70842025-10-03T21:19:10.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70852025-10-03T21:19:10.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
70862025-10-03T21:19:10.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70872025-10-03T21:19:10.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70882025-10-03T21:19:10.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
70892025-10-03T21:19:10.340ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70902025-10-03T21:19:10.340ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
70912025-10-03T21:19:10.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
70922025-10-03T21:19:10.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
70932025-10-03T21:19:10.348ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70942025-10-03T21:19:10.348ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70952025-10-03T21:19:10.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70962025-10-03T21:19:10.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70972025-10-03T21:19:10.349ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
70982025-10-03T21:19:10.349ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 156.321515ms iteration = 73 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
70992025-10-03T21:19:10.349ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
71002025-10-03T21:19:10.349ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
71012025-10-03T21:19:10.349ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71022025-10-03T21:19:10.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71032025-10-03T21:19:10.352ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71042025-10-03T21:19:10.354ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71052025-10-03T21:19:10.355ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 161.955309ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
71062025-10-03T21:19:10.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71072025-10-03T21:19:10.356ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71082025-10-03T21:19:10.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71092025-10-03T21:19:10.358ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
71102025-10-03T21:19:10.359ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 165.563679ms iteration = 37 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
71112025-10-03T21:19:10.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
71122025-10-03T21:19:10.360ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71132025-10-03T21:19:10.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71142025-10-03T21:19:10.362ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71152025-10-03T21:19:10.362ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
71162025-10-03T21:19:10.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
71172025-10-03T21:19:10.364ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
71182025-10-03T21:19:10.364ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
71192025-10-03T21:19:10.364ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-10-03 21:07:14.470425 UTC dns_group = external generation = 2
71202025-10-03T21:19:10.364ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 171.236233ms iteration = 13 status = {"generation":2}
71212025-10-03T21:19:10.365ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 171.741258ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
71222025-10-03T21:19:10.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71232025-10-03T21:19:10.366ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
71242025-10-03T21:19:10.366ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71252025-10-03T21:19:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71262025-10-03T21:19:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71272025-10-03T21:19:10.368ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 174.650042ms iteration = 39 status = {"physical_disks_added":0}
71282025-10-03T21:19:10.368ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
71292025-10-03T21:19:10.368ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 175.548302ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
71302025-10-03T21:19:10.368ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
71312025-10-03T21:19:10.368ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 175.744744ms iteration = 145 status = {"config_updated":false}
71322025-10-03T21:19:10.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71332025-10-03T21:19:10.373ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
71342025-10-03T21:19:10.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71352025-10-03T21:19:10.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71362025-10-03T21:19:10.376ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 182.629111ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
71372025-10-03T21:19:10.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71382025-10-03T21:19:10.379ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
71392025-10-03T21:19:10.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71402025-10-03T21:19:10.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71412025-10-03T21:19:10.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71422025-10-03T21:19:10.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71432025-10-03T21:19:10.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71442025-10-03T21:19:10.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
71452025-10-03T21:19:10.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71462025-10-03T21:19:10.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71472025-10-03T21:19:10.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71482025-10-03T21:19:10.396ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
71492025-10-03T21:19:10.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71502025-10-03T21:19:10.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71512025-10-03T21:19:10.405ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
71522025-10-03T21:19:10.405ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
71532025-10-03T21:19:10.405ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 211.649699ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
71542025-10-03T21:19:10.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71552025-10-03T21:19:10.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71562025-10-03T21:19:10.410ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71572025-10-03T21:19:10.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71582025-10-03T21:19:10.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71592025-10-03T21:19:10.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71602025-10-03T21:19:10.412ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
71612025-10-03T21:19:10.412ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
71622025-10-03T21:19:10.412ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 219.192854ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
71632025-10-03T21:19:10.414ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71642025-10-03T21:19:10.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71652025-10-03T21:19:10.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71662025-10-03T21:19:10.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
71672025-10-03T21:19:10.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71682025-10-03T21:19:10.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71692025-10-03T21:19:10.421ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
71702025-10-03T21:19:10.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71712025-10-03T21:19:10.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
71722025-10-03T21:19:10.422ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71732025-10-03T21:19:10.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71742025-10-03T21:19:10.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71752025-10-03T21:19:10.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
71762025-10-03T21:19:10.430ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71772025-10-03T21:19:10.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71782025-10-03T21:19:10.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
71792025-10-03T21:19:10.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71802025-10-03T21:19:10.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71812025-10-03T21:19:10.432ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 239.006078ms iteration = 25 status = 0
71822025-10-03T21:19:10.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71832025-10-03T21:19:10.435ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71842025-10-03T21:19:10.440ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
71852025-10-03T21:19:10.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71862025-10-03T21:19:10.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71872025-10-03T21:19:10.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
71882025-10-03T21:19:10.441ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71892025-10-03T21:19:10.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
71902025-10-03T21:19:10.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71912025-10-03T21:19:10.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71922025-10-03T21:19:10.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71932025-10-03T21:19:10.447ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
71942025-10-03T21:19:10.450ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
71952025-10-03T21:19:10.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
71962025-10-03T21:19:10.451ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
71972025-10-03T21:19:10.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71982025-10-03T21:19:10.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71992025-10-03T21:19:10.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72002025-10-03T21:19:10.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72012025-10-03T21:19:10.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72022025-10-03T21:19:10.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72032025-10-03T21:19:10.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72042025-10-03T21:19:10.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
72052025-10-03T21:19:10.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72062025-10-03T21:19:10.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72072025-10-03T21:19:10.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72082025-10-03T21:19:10.469ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
72092025-10-03T21:19:10.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72102025-10-03T21:19:10.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72112025-10-03T21:19:10.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72122025-10-03T21:19:10.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72132025-10-03T21:19:10.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72142025-10-03T21:19:10.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72152025-10-03T21:19:10.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72162025-10-03T21:19:10.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
72172025-10-03T21:19:10.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72182025-10-03T21:19:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72192025-10-03T21:19:10.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72202025-10-03T21:19:10.490ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
72212025-10-03T21:19:10.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72222025-10-03T21:19:10.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72232025-10-03T21:19:10.491ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = DeployNexusRecords step id = Ensure
72242025-10-03T21:19:10.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
72252025-10-03T21:19:10.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
72262025-10-03T21:19:10.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72272025-10-03T21:19:10.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72282025-10-03T21:19:10.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72292025-10-03T21:19:10.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72302025-10-03T21:19:10.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72312025-10-03T21:19:10.497ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
72322025-10-03T21:19:10.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
72332025-10-03T21:19:10.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
72342025-10-03T21:19:10.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72352025-10-03T21:19:10.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
72362025-10-03T21:19:10.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
72372025-10-03T21:19:10.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72382025-10-03T21:19:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
72392025-10-03T21:19:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
72402025-10-03T21:19:10.506ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
72412025-10-03T21:19:10.506ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72422025-10-03T21:19:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 314.073135ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
72432025-10-03T21:19:10.507ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "315cd393-5089-4386-b674-f5a11316c10e", "content-length": "26", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} })
72442025-10-03T21:19:10.507ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72452025-10-03T21:19:10.507ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
72462025-10-03T21:19:10.509ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
72472025-10-03T21:19:10.510ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
72482025-10-03T21:19:10.510ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72492025-10-03T21:19:10.510ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "56a82229-6caa-4f3e-a775-9235e40dccf8", "content-length": "26", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} })
72502025-10-03T21:19:10.510ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72512025-10-03T21:19:10.515ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
72522025-10-03T21:19:10.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72532025-10-03T21:19:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
72542025-10-03T21:19:10.523ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72552025-10-03T21:19:10.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
72562025-10-03T21:19:10.526ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72572025-10-03T21:19:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
72582025-10-03T21:19:10.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72592025-10-03T21:19:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
72602025-10-03T21:19:10.534ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
72612025-10-03T21:19:10.534ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
72622025-10-03T21:19:10.535ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e28a577-daf5-4e27-b6c0-fac8544d32e5", "content-length": "372", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} })
72632025-10-03T21:19:10.535ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72642025-10-03T21:19:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72652025-10-03T21:19:10.540ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72662025-10-03T21:19:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
72672025-10-03T21:19:10.543ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72682025-10-03T21:19:10.546ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
72692025-10-03T21:19:10.546ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72702025-10-03T21:19:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
72712025-10-03T21:19:10.548ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
72722025-10-03T21:19:10.548ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72732025-10-03T21:19:10.549ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "029eb7fe-7149-44f3-a4aa-b8764aa5d2e8", "content-length": "26", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} })
72742025-10-03T21:19:10.549ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72752025-10-03T21:19:10.549ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72762025-10-03T21:19:10.552ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
72772025-10-03T21:19:10.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
72782025-10-03T21:19:10.552ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72792025-10-03T21:19:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
72802025-10-03T21:19:10.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72812025-10-03T21:19:10.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72822025-10-03T21:19:10.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72832025-10-03T21:19:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72842025-10-03T21:19:10.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72852025-10-03T21:19:10.574ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = SledAgent step id = Ensure
72862025-10-03T21:19:10.580ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
72872025-10-03T21:19:10.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72882025-10-03T21:19:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72892025-10-03T21:19:10.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72902025-10-03T21:19:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72912025-10-03T21:19:10.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72922025-10-03T21:19:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
72932025-10-03T21:19:10.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72942025-10-03T21:19:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72952025-10-03T21:19:10.596ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-10-03T21:07:04.185535Z, time_modified: 2025-10-03T21:07:04.185535Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
72962025-10-03T21:19:10.596ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 403.866038ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]}},"default_endpoint":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
72972025-10-03T21:19:10.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72982025-10-03T21:19:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72992025-10-03T21:19:10.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73002025-10-03T21:19:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
73012025-10-03T21:19:10.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73022025-10-03T21:19:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73032025-10-03T21:19:10.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73042025-10-03T21:19:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73052025-10-03T21:19:10.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73062025-10-03T21:19:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
73072025-10-03T21:19:10.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73082025-10-03T21:19:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73092025-10-03T21:19:10.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73102025-10-03T21:19:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73112025-10-03T21:19:10.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73122025-10-03T21:19:10.621ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
73132025-10-03T21:19:10.621ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
73142025-10-03T21:19:10.621ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e90c6d3e-d6a5-4fb6-bd06-d561a4108a54", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} })
73152025-10-03T21:19:10.621ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73162025-10-03T21:19:10.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73172025-10-03T21:19:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
73182025-10-03T21:19:10.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73192025-10-03T21:19:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73202025-10-03T21:19:10.625ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73212025-10-03T21:19:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73222025-10-03T21:19:10.628ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73232025-10-03T21:19:10.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73242025-10-03T21:19:10.632ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
73252025-10-03T21:19:10.632ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
73262025-10-03T21:19:10.633ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1960779d-f2ed-4dca-80e0-e1cfa8944cb1", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} })
73272025-10-03T21:19:10.635ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 442.031329ms iteration = 25 status = {}
73282025-10-03T21:19:10.657ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 465.11014ms iteration = 25 status = {}
73292025-10-03T21:19:10.666ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
73302025-10-03T21:19:10.678ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
73312025-10-03T21:19:10.678ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 484.911293ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
73322025-10-03T21:19:10.680ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
73332025-10-03T21:19:10.680ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
73342025-10-03T21:19:10.680ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = b41372f7-82c0-487c-9c54-2cd9cec92e1e parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
73352025-10-03T21:19:10.680ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
73362025-10-03T21:19:10.680ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 487.529713ms iteration = 29 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
73372025-10-03T21:19:10.777ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
73382025-10-03T21:19:10.777ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "fadb4737-aad1-490e-b030-58ac6a66b2ee", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73392025-10-03T21:19:10.777ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73402025-10-03T21:19:10.779ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "780fcd13-86c6-4883-b213-f21a591131d7", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73412025-10-03T21:19:10.779ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73422025-10-03T21:19:10.782ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73432025-10-03T21:19:10.782ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 589.926329ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
73442025-10-03T21:19:10.811ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
73452025-10-03T21:19:10.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
73462025-10-03T21:19:10.814ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 621.143991ms iteration = 25 status = null
73472025-10-03T21:19:10.833ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
73482025-10-03T21:19:10.834ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "b9e02f3b-bd8d-4c74-8436-9d9ddd5a0a27", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73492025-10-03T21:19:10.834ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73502025-10-03T21:19:10.836ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1bf58909-2f7d-4a6b-adf4-cb31c055ca40", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73512025-10-03T21:19:10.836ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73522025-10-03T21:19:10.839ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "f9d846fb-4d04-47da-8694-0f9fed4f4052", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73532025-10-03T21:19:10.839ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73542025-10-03T21:19:10.840ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
73552025-10-03T21:19:10.841ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73562025-10-03T21:19:10.841ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 648.857484ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
73572025-10-03T21:19:10.842ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "44c4aa8c-4145-4583-8e6e-f53045aaa23e", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} }) sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
73582025-10-03T21:19:10.845ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = FirewallRules step id = Ensure
73592025-10-03T21:19:10.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73602025-10-03T21:19:10.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73612025-10-03T21:19:10.846ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
73622025-10-03T21:19:10.847ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a26dbfef-419c-4cb6-8eb0-971813d8ccc0", "content-length": "691", "date": "Fri, 03 Oct 2025 21:19:10 GMT"} })
73632025-10-03T21:19:10.847ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
73642025-10-03T21:19:10.847ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
73652025-10-03T21:19:10.847ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
73662025-10-03T21:19:10.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73672025-10-03T21:19:10.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73682025-10-03T21:19:10.849ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 656.453349ms iteration = 25 status = {}
73692025-10-03T21:19:10.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73702025-10-03T21:19:10.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73712025-10-03T21:19:10.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73722025-10-03T21:19:10.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73732025-10-03T21:19:10.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73742025-10-03T21:19:10.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73752025-10-03T21:19:10.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73762025-10-03T21:19:10.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73772025-10-03T21:19:10.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73782025-10-03T21:19:10.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
73792025-10-03T21:19:10.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73802025-10-03T21:19:10.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73812025-10-03T21:19:10.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73822025-10-03T21:19:10.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73832025-10-03T21:19:10.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73842025-10-03T21:19:10.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73852025-10-03T21:19:10.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73862025-10-03T21:19:10.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
73872025-10-03T21:19:10.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73882025-10-03T21:19:10.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73892025-10-03T21:19:10.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73902025-10-03T21:19:10.878ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73912025-10-03T21:19:10.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73922025-10-03T21:19:10.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73932025-10-03T21:19:10.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73942025-10-03T21:19:10.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73952025-10-03T21:19:10.881ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
73962025-10-03T21:19:10.881ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73972025-10-03T21:19:10.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73982025-10-03T21:19:10.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73992025-10-03T21:19:10.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74002025-10-03T21:19:10.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74012025-10-03T21:19:10.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74022025-10-03T21:19:10.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74032025-10-03T21:19:10.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74042025-10-03T21:19:10.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74052025-10-03T21:19:10.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74062025-10-03T21:19:10.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74072025-10-03T21:19:10.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
74082025-10-03T21:19:10.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74092025-10-03T21:19:10.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74102025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
74112025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
74122025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
74132025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
74142025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74152025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74162025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74172025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74182025-10-03T21:19:10.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74192025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74202025-10-03T21:19:10.896ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74212025-10-03T21:19:10.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74222025-10-03T21:19:10.897ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
74232025-10-03T21:19:10.897ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74242025-10-03T21:19:10.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74252025-10-03T21:19:10.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74262025-10-03T21:19:10.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74272025-10-03T21:19:10.904ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
74282025-10-03T21:19:10.904ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
74292025-10-03T21:19:10.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74302025-10-03T21:19:10.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74312025-10-03T21:19:10.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74322025-10-03T21:19:10.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74332025-10-03T21:19:10.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74342025-10-03T21:19:10.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74352025-10-03T21:19:10.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74362025-10-03T21:19:10.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74372025-10-03T21:19:10.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74382025-10-03T21:19:10.916ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
74392025-10-03T21:19:10.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74402025-10-03T21:19:10.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74412025-10-03T21:19:10.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74422025-10-03T21:19:10.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74432025-10-03T21:19:10.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74442025-10-03T21:19:10.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74452025-10-03T21:19:10.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74462025-10-03T21:19:10.925ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
74472025-10-03T21:19:10.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74482025-10-03T21:19:10.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74492025-10-03T21:19:10.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74502025-10-03T21:19:10.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74512025-10-03T21:19:10.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74522025-10-03T21:19:10.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74532025-10-03T21:19:10.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74542025-10-03T21:19:10.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74552025-10-03T21:19:10.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74562025-10-03T21:19:10.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74572025-10-03T21:19:10.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74582025-10-03T21:19:10.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74592025-10-03T21:19:10.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74602025-10-03T21:19:10.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74612025-10-03T21:19:10.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74622025-10-03T21:19:10.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74632025-10-03T21:19:10.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74642025-10-03T21:19:10.949ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
74652025-10-03T21:19:10.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74662025-10-03T21:19:10.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74672025-10-03T21:19:10.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74682025-10-03T21:19:10.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74692025-10-03T21:19:10.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74702025-10-03T21:19:10.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74712025-10-03T21:19:10.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74722025-10-03T21:19:10.958ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
74732025-10-03T21:19:10.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74742025-10-03T21:19:10.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74752025-10-03T21:19:10.965ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
74762025-10-03T21:19:10.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74772025-10-03T21:19:10.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74782025-10-03T21:19:10.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74792025-10-03T21:19:10.969ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
74802025-10-03T21:19:10.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74812025-10-03T21:19:10.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74822025-10-03T21:19:10.971ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
74832025-10-03T21:19:10.971ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
74842025-10-03T21:19:10.971ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
74852025-10-03T21:19:11.036ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), time_created: 2025-10-03T21:07:05.994Z, time_modified: 2025-10-03T21:07:05.994Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
74862025-10-03T21:19:11.036ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
74872025-10-03T21:19:11.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74882025-10-03T21:19:11.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74892025-10-03T21:19:11.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74902025-10-03T21:19:11.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74912025-10-03T21:19:11.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74922025-10-03T21:19:11.074ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
74932025-10-03T21:19:11.074ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
74942025-10-03T21:19:11.075ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "078065fd-0dc1-4a5d-af4c-21bb9b7a8d5d", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
74952025-10-03T21:19:11.076ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
74962025-10-03T21:19:11.076ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = Dns step id = Ensure
74972025-10-03T21:19:11.077ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
74982025-10-03T21:19:11.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74992025-10-03T21:19:11.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75002025-10-03T21:19:11.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75012025-10-03T21:19:11.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75022025-10-03T21:19:11.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75032025-10-03T21:19:11.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
75042025-10-03T21:19:11.120ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
75052025-10-03T21:19:11.120ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
75062025-10-03T21:19:11.120ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
75072025-10-03T21:19:11.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75082025-10-03T21:19:11.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75092025-10-03T21:19:11.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75102025-10-03T21:19:11.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
75112025-10-03T21:19:11.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75122025-10-03T21:19:11.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75132025-10-03T21:19:11.127ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
75142025-10-03T21:19:11.127ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
75152025-10-03T21:19:11.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75162025-10-03T21:19:11.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75172025-10-03T21:19:11.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75182025-10-03T21:19:11.132ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
75192025-10-03T21:19:11.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75202025-10-03T21:19:11.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75212025-10-03T21:19:11.135ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
75222025-10-03T21:19:11.135ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
75232025-10-03T21:19:11.135ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
75242025-10-03T21:19:11.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75252025-10-03T21:19:11.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75262025-10-03T21:19:11.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75272025-10-03T21:19:11.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75282025-10-03T21:19:11.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75292025-10-03T21:19:11.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
75302025-10-03T21:19:11.141ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
75312025-10-03T21:19:11.141ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
75322025-10-03T21:19:11.141ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
75332025-10-03T21:19:11.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75342025-10-03T21:19:11.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75352025-10-03T21:19:11.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75362025-10-03T21:19:11.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
75372025-10-03T21:19:11.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75382025-10-03T21:19:11.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75392025-10-03T21:19:11.148ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
75402025-10-03T21:19:11.148ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
75412025-10-03T21:19:11.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75422025-10-03T21:19:11.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75432025-10-03T21:19:11.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75442025-10-03T21:19:11.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
75452025-10-03T21:19:11.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75462025-10-03T21:19:11.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75472025-10-03T21:19:11.154ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
75482025-10-03T21:19:11.154ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
75492025-10-03T21:19:11.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75502025-10-03T21:19:11.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75512025-10-03T21:19:11.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75522025-10-03T21:19:11.158ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
75532025-10-03T21:19:11.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75542025-10-03T21:19:11.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75552025-10-03T21:19:11.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75562025-10-03T21:19:11.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75572025-10-03T21:19:11.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75582025-10-03T21:19:11.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75592025-10-03T21:19:11.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75602025-10-03T21:19:11.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
75612025-10-03T21:19:11.167ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
75622025-10-03T21:19:11.167ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
75632025-10-03T21:19:11.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = OmicronZones step id = Cleanup
75642025-10-03T21:19:11.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = OmicronZones step id = Cleanup
75652025-10-03T21:19:11.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = PhysicalDisks step id = Cleanup
75662025-10-03T21:19:11.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = Clickhouse step id = Ensure
75672025-10-03T21:19:11.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = Clickhouse step id = Ensure
75682025-10-03T21:19:11.198ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
75692025-10-03T21:19:11.203ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "48b4ae95-0793-4301-ba17-9a7210f90bf7", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
75702025-10-03T21:19:11.204ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = SupportBundles step id = Cleanup
75712025-10-03T21:19:11.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75722025-10-03T21:19:11.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75732025-10-03T21:19:11.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75742025-10-03T21:19:11.208ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
75752025-10-03T21:19:11.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75762025-10-03T21:19:11.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75772025-10-03T21:19:11.209ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "13cc50f5-83bc-4ea2-94c0-ba27ea25ead8", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
75782025-10-03T21:19:11.209ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
75792025-10-03T21:19:11.209ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
75802025-10-03T21:19:11.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75812025-10-03T21:19:11.210ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "39c89cea-7dd3-469c-8f66-203d691aaea4", "content-length": "682", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
75822025-10-03T21:19:11.210ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75832025-10-03T21:19:11.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75842025-10-03T21:19:11.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75852025-10-03T21:19:11.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75862025-10-03T21:19:11.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75872025-10-03T21:19:11.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75882025-10-03T21:19:11.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
75892025-10-03T21:19:11.216ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75902025-10-03T21:19:11.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75912025-10-03T21:19:11.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75922025-10-03T21:19:11.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75932025-10-03T21:19:11.226ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 405 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = ece25cb2-d335-4fd0-ba6e-a69004238ca5 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
75942025-10-03T21:19:11.226ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.033493156s iteration = 25 status = {}
75952025-10-03T21:19:11.252ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = OmicronZones step id = Cleanup
75962025-10-03T21:19:11.252ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
75972025-10-03T21:19:11.252ZDEBGnexus (BackgroundTasks): re-assign sagas: found Nexus instances background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
75982025-10-03T21:19:11.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75992025-10-03T21:19:11.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76002025-10-03T21:19:11.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76012025-10-03T21:19:11.255ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
76022025-10-03T21:19:11.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76032025-10-03T21:19:11.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76042025-10-03T21:19:11.289ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:54 nexus_zone_ids = []
76052025-10-03T21:19:11.289ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(675bea75-8e2d-4986-86f2-b590a30bb084 (blueprint), false)
76062025-10-03T21:19:11.289ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = Cockroach step id = Ensure
76072025-10-03T21:19:11.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76082025-10-03T21:19:11.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76092025-10-03T21:19:11.302ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
76102025-10-03T21:19:11.302ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 06910a0d-cc6f-40a5-88a3-46c68a231e27 step component = MgsUpdates step id = Ensure
76112025-10-03T21:19:11.302ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 38 reason = Dependency
76122025-10-03T21:19:11.302ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Dependency
76132025-10-03T21:19:11.302ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76142025-10-03T21:19:11.302ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76152025-10-03T21:19:11.302ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.109822868s iteration = 14 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3440,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":123819478,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":123819438,"secs":0}},"event_index":1,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":123837338,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":174388139,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":174388099,"secs":0}},"event_index":2,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":298252667,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":83273339,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":83273289,"secs":0}},"event_index":3,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":381540426,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":270828758,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":270828718,"secs":0}},"event_index":4,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":652385144,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":231709605,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":231709575,"secs":0}},"event_index":5,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":884110979,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":90933526,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":90933486,"secs":0}},"event_index":6,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":975053966,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3950,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3910,"secs":0}},"event_index":7,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":975068756,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2260,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2220,"secs":0}},"event_index":8,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":975073206,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2760,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2720,"secs":0}},"event_index":9,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":975077446,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2160,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2120,"secs":0}},"event_index":10,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":975081346,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36310230,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36310180,"secs":0}},"event_index":11,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":11392756,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47875631,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":47875601,"secs":0}},"event_index":12,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":59279577,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36853436,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":36853386,"secs":0}},"event_index":13,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":96151493,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":13083118,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":13083078,"secs":0}},"event_index":14,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":109243801,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5900,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5860,"secs":0}},"event_index":15,"execution_id":"cf827ca9-85dd-4ae1-aa64-e583f6728a6b","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":109256381,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"675bea75-8e2d-4986-86f2-b590a30bb084"}
76162025-10-03T21:19:11.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
76172025-10-03T21:19:11.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76182025-10-03T21:19:11.305ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76192025-10-03T21:19:11.305ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76202025-10-03T21:19:11.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76212025-10-03T21:19:11.308ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76222025-10-03T21:19:11.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76232025-10-03T21:19:11.308ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
76242025-10-03T21:19:11.308ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76252025-10-03T21:19:11.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
76262025-10-03T21:19:11.310ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76272025-10-03T21:19:11.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76282025-10-03T21:19:11.311ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76292025-10-03T21:19:11.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76302025-10-03T21:19:11.313ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76312025-10-03T21:19:11.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76322025-10-03T21:19:11.315ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
76332025-10-03T21:19:11.315ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 13.086078ms iteration = 38 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
76342025-10-03T21:19:11.342ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(df189a1a-0917-4749-bff1-6603c1d9275e, 0), (8e77a9b0-59eb-492d-aaf5-16d924b6dcdc, 0), (7cc7d81f-8f13-4a68-85a6-1781b01fb64f, 0), (c5bd80c2-eb4a-4bc4-80c2-5566402782b5, 0)] collection_id = df189a1a-0917-4749-bff1-6603c1d9275e
76352025-10-03T21:19:11.342ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76362025-10-03T21:19:11.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
76372025-10-03T21:19:11.454ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = df189a1a-0917-4749-bff1-6603c1d9275e file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
76382025-10-03T21:19:11.454ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76392025-10-03T21:19:11.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
76402025-10-03T21:19:11.459ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(8e77a9b0-59eb-492d-aaf5-16d924b6dcdc, 0), (7cc7d81f-8f13-4a68-85a6-1781b01fb64f, 0), (c5bd80c2-eb4a-4bc4-80c2-5566402782b5, 0), (ce03add5-1108-49b7-ab5b-baeeab316a5f, 0)] collection_id = 8e77a9b0-59eb-492d-aaf5-16d924b6dcdc
76412025-10-03T21:19:11.459ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76422025-10-03T21:19:11.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
76432025-10-03T21:19:11.591ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 8e77a9b0-59eb-492d-aaf5-16d924b6dcdc file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
76442025-10-03T21:19:11.591ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76452025-10-03T21:19:11.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
76462025-10-03T21:19:11.595ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(7cc7d81f-8f13-4a68-85a6-1781b01fb64f, 0), (c5bd80c2-eb4a-4bc4-80c2-5566402782b5, 0), (ce03add5-1108-49b7-ab5b-baeeab316a5f, 0)]
76472025-10-03T21:19:11.595ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5197, tv_nsec: 82363055 } })
76482025-10-03T21:19:11.627ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("26cdd4d2-d0ac-43df-bf3e-350fd160efec.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41cf4311-4781-41d3-b8bc-181c344891b0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b55c423c-6cde-4185-879d-3a1c4cf01de4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b807e991-1a36-4877-8937-3efb3994dbc2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::3)) }], valid_until: Instant { tv_sec: 5197, tv_nsec: 114566059 } })
76492025-10-03T21:19:11.628ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
76502025-10-03T21:19:11.628ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
76512025-10-03T21:19:11.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
76522025-10-03T21:19:11.629ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2517e862-3091-488f-8cab-1ff3100d6958", "content-length": "476", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76532025-10-03T21:19:11.629ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
76542025-10-03T21:19:11.629ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e85b13d-af72-41c3-89f3-3e1157bf365e", "content-length": "734", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76552025-10-03T21:19:11.629ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
76562025-10-03T21:19:11.630ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "547cf345-ca38-4835-b58f-e7b2b2b01b76", "content-length": "111", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76572025-10-03T21:19:11.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
76582025-10-03T21:19:11.630ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf318900-5efa-40d5-ac8e-af02c5c470ea", "content-length": "111", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76592025-10-03T21:19:11.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
76602025-10-03T21:19:11.631ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2628cc03-0e03-4a25-ab55-ed1a25ff56f5", "content-length": "163", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76612025-10-03T21:19:11.631ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
76622025-10-03T21:19:11.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18aea28d-6424-4a12-84d2-36f6a04f1c48", "content-length": "163", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76632025-10-03T21:19:11.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
76642025-10-03T21:19:11.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeee2bb1-f5b3-4730-9458-da03ab40b70e", "content-length": "166", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76652025-10-03T21:19:11.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
76662025-10-03T21:19:11.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf8f3540-c27e-4bdc-978d-69a8e5c614b5", "content-length": "166", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76672025-10-03T21:19:11.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
76682025-10-03T21:19:11.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9315445d-9679-475b-82e5-3d7db22881fb", "content-length": "702", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76692025-10-03T21:19:11.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
76702025-10-03T21:19:11.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f330732-dec2-43ae-a719-cb96e12d0b17", "content-length": "727", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76712025-10-03T21:19:11.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
76722025-10-03T21:19:11.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "716f5e84-e7ac-4764-b3ee-d53d5072dd85", "content-length": "729", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76732025-10-03T21:19:11.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
76742025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "04bda5c9-2cc8-453c-8d60-3e00e56d8fd7", "content-length": "728", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76752025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
76762025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "567c5f71-6095-4f97-b11d-10692f352c2c", "content-length": "722", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76772025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
76782025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "b332b905-947e-4ea5-89e1-9acdf9c5bc21", "content-length": "10", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76792025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
76802025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "83966f47-0ecf-41eb-9df2-76eb2c4ae68e", "content-length": "144", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76812025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
76822025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "48f27bd9-69ef-443c-bccb-0932269e2680", "content-length": "147", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76832025-10-03T21:19:11.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
76842025-10-03T21:19:11.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b78b5b1e-ef57-4407-be0a-86d68434e6ca", "content-length": "109", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76852025-10-03T21:19:11.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
76862025-10-03T21:19:11.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1688e51-7d8e-4072-9f3c-22016ac09553", "content-length": "109", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76872025-10-03T21:19:11.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
76882025-10-03T21:19:11.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "79f38738-1ed3-4b27-8a52-dd102fff9416", "content-length": "163", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76892025-10-03T21:19:11.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
76902025-10-03T21:19:11.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c20fcf5d-fffc-4d88-a09b-bd0cd49e4b4e", "content-length": "163", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76912025-10-03T21:19:11.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
76922025-10-03T21:19:11.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8130114-9ca6-41e2-90cb-aaf77540e967", "content-length": "166", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76932025-10-03T21:19:11.637ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
76942025-10-03T21:19:11.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce578d2e-0fa4-410f-a4a9-f425afb24fe9", "content-length": "166", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76952025-10-03T21:19:11.637ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
76962025-10-03T21:19:11.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "534f06c8-e554-484b-b2cc-e053c8a64a36", "content-length": "702", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76972025-10-03T21:19:11.638ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
76982025-10-03T21:19:11.638ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b65d5a0-56af-41a4-8a6f-91947e320755", "content-length": "727", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
76992025-10-03T21:19:11.638ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
77002025-10-03T21:19:11.638ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09264d26-d245-4ee1-b368-48fee9144b31", "content-length": "729", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77012025-10-03T21:19:11.638ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
77022025-10-03T21:19:11.638ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee2c2a54-c5c5-464d-982f-f1747f523d2f", "content-length": "728", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77032025-10-03T21:19:11.638ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77042025-10-03T21:19:11.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
77052025-10-03T21:19:11.640ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77062025-10-03T21:19:11.642ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
77072025-10-03T21:19:11.642ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77082025-10-03T21:19:11.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
77092025-10-03T21:19:11.644ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77102025-10-03T21:19:11.645ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
77112025-10-03T21:19:11.645ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77122025-10-03T21:19:11.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
77132025-10-03T21:19:11.678ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
77142025-10-03T21:19:11.678ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
77152025-10-03T21:19:11.796ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "242ea1d1-ca3e-4684-b5b8-3117ed7bea87", "content-length": "88862", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77162025-10-03T21:19:11.798ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
77172025-10-03T21:19:11.798ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
77182025-10-03T21:19:11.798ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
77192025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
77202025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
77212025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
77222025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
77232025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
77242025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f32c6340-ad82-4399-b8f5-ed5cff3b6030", "content-length": "64", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77252025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
77262025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c69997c2-f3fb-4f07-9817-d95a834f0f3f", "content-length": "64", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77272025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "29509672-e3ab-430d-ae2f-2c6612d56577", "content-length": "64", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77282025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
77292025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
77302025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "36c39747-ea77-4aac-835a-d14314a869c2", "content-length": "64", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77312025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
77322025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "945c1fa5-9ea7-49d1-bf0f-40f015b92481", "content-length": "64", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77332025-10-03T21:19:11.798ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
77342025-10-03T21:19:11.838ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8aac943a-dd9f-4245-8e88-6aa25700ddcb", "content-length": "410460", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77352025-10-03T21:19:11.840ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2c97567-ac5b-4d25-8a36-27b8816cfa0b", "content-length": "409676", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77362025-10-03T21:19:11.840ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d8d42fa-e079-498d-b521-1988f98f764e", "content-length": "434144", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77372025-10-03T21:19:11.846ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "87dafee1-21d8-4f5f-8f39-9a4f291305ab", "content-length": "400377", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77382025-10-03T21:19:11.865ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e4da09c-c70b-4f82-a6dc-ac49343634a1", "content-length": "407787", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77392025-10-03T21:19:11.867ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
77402025-10-03T21:19:11.867ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
77412025-10-03T21:19:11.867ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
77422025-10-03T21:19:11.867ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
77432025-10-03T21:19:11.867ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
77442025-10-03T21:19:11.898ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 (service)
77452025-10-03T21:19:11.898ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
77462025-10-03T21:19:11.900ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "f752c5fd-e75c-41ec-9f41-65af56c324ed", "content-length": "122", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77472025-10-03T21:19:11.901ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
77482025-10-03T21:19:11.990ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
77492025-10-03T21:19:11.990ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
77502025-10-03T21:19:11.990ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b44fc18-6e1d-4dbe-8f69-f4f7ec12dec6", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77512025-10-03T21:19:11.991ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
77522025-10-03T21:19:11.993ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
77532025-10-03T21:19:11.993ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
77542025-10-03T21:19:11.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "671e97c7-62ad-41a8-863b-02c774427e80", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77552025-10-03T21:19:11.994ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
77562025-10-03T21:19:11.996ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
77572025-10-03T21:19:11.996ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
77582025-10-03T21:19:11.996ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "766706ae-19bd-47a5-90ea-6d88a05b9a66", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:19:11 GMT"} })
77592025-10-03T21:19:11.996ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
77602025-10-03T21:19:11.998ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
77612025-10-03T21:19:11.998ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
77622025-10-03T21:19:12.004ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77632025-10-03T21:19:12.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
77642025-10-03T21:19:12.006ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77652025-10-03T21:19:12.009ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
77662025-10-03T21:19:12.009ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77672025-10-03T21:19:12.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
77682025-10-03T21:19:12.209ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 69ec4344-f92b-4710-adb9-58137efc8a31 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
77692025-10-03T21:19:12.209ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 69ec4344-f92b-4710-adb9-58137efc8a31 time_started = 2025-10-03 21:19:11.628685 UTC
77702025-10-03T21:19:12.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 30 reason = Dependency
77712025-10-03T21:19:12.209ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 40 reason = Dependency
77722025-10-03T21:19:12.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
77732025-10-03T21:19:12.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77742025-10-03T21:19:12.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 18 reason = Dependency
77752025-10-03T21:19:12.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
77762025-10-03T21:19:12.209ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 907.475744ms iteration = 15 status = {"collection_id":"69ec4344-f92b-4710-adb9-58137efc8a31","time_done":"2025-10-03 21:19:11.998941 UTC","time_started":"2025-10-03 21:19:11.628685 UTC"}
77772025-10-03T21:19:12.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
77782025-10-03T21:19:12.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77792025-10-03T21:19:12.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
77802025-10-03T21:19:12.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77812025-10-03T21:19:12.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77822025-10-03T21:19:12.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
77832025-10-03T21:19:12.215ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77842025-10-03T21:19:12.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77852025-10-03T21:19:12.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
77862025-10-03T21:19:12.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
77872025-10-03T21:19:12.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
77882025-10-03T21:19:12.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
77892025-10-03T21:19:12.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
77902025-10-03T21:19:12.220ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77912025-10-03T21:19:12.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77922025-10-03T21:19:12.222ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77932025-10-03T21:19:12.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
77942025-10-03T21:19:12.225ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77952025-10-03T21:19:12.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77962025-10-03T21:19:12.231ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.872937ms iteration = 40 status = {"physical_disks_added":0}
77972025-10-03T21:19:12.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
77982025-10-03T21:19:12.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
77992025-10-03T21:19:12.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78002025-10-03T21:19:12.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
78012025-10-03T21:19:12.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78022025-10-03T21:19:12.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78032025-10-03T21:19:12.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
78042025-10-03T21:19:12.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
78052025-10-03T21:19:12.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78062025-10-03T21:19:12.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78072025-10-03T21:19:12.261ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
78082025-10-03T21:19:12.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78092025-10-03T21:19:12.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78102025-10-03T21:19:12.262ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
78112025-10-03T21:19:12.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
78122025-10-03T21:19:12.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
78132025-10-03T21:19:12.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78142025-10-03T21:19:12.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78152025-10-03T21:19:12.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78162025-10-03T21:19:12.265ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
78172025-10-03T21:19:12.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
78182025-10-03T21:19:12.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
78192025-10-03T21:19:12.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78202025-10-03T21:19:12.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
78212025-10-03T21:19:12.267ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
78222025-10-03T21:19:12.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78232025-10-03T21:19:12.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78242025-10-03T21:19:12.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
78252025-10-03T21:19:12.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78262025-10-03T21:19:12.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78272025-10-03T21:19:12.271ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
78282025-10-03T21:19:12.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78292025-10-03T21:19:12.271ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
78302025-10-03T21:19:12.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
78312025-10-03T21:19:12.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78322025-10-03T21:19:12.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
78332025-10-03T21:19:12.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78342025-10-03T21:19:12.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78352025-10-03T21:19:12.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78362025-10-03T21:19:12.275ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
78372025-10-03T21:19:12.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
78382025-10-03T21:19:12.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
78392025-10-03T21:19:12.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78402025-10-03T21:19:12.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78412025-10-03T21:19:12.277ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
78422025-10-03T21:19:12.277ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 67.668454ms iteration = 18 status = {"blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","inventory_collection_id":"32405174-c122-4568-b76b-b784f3b35147","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
78432025-10-03T21:19:12.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78442025-10-03T21:19:12.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
78452025-10-03T21:19:12.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78462025-10-03T21:19:12.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78472025-10-03T21:19:12.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78482025-10-03T21:19:12.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78492025-10-03T21:19:12.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78502025-10-03T21:19:12.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
78512025-10-03T21:19:12.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78522025-10-03T21:19:12.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78532025-10-03T21:19:12.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78542025-10-03T21:19:12.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
78552025-10-03T21:19:12.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78562025-10-03T21:19:12.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78572025-10-03T21:19:12.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78582025-10-03T21:19:12.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78592025-10-03T21:19:12.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78602025-10-03T21:19:12.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
78612025-10-03T21:19:12.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78622025-10-03T21:19:12.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78632025-10-03T21:19:12.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78642025-10-03T21:19:12.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78652025-10-03T21:19:12.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78662025-10-03T21:19:12.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78672025-10-03T21:19:12.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78682025-10-03T21:19:12.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
78692025-10-03T21:19:12.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78702025-10-03T21:19:12.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78712025-10-03T21:19:12.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78722025-10-03T21:19:12.309ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
78732025-10-03T21:19:12.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78742025-10-03T21:19:12.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78752025-10-03T21:19:12.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78762025-10-03T21:19:12.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78772025-10-03T21:19:12.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78782025-10-03T21:19:12.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78792025-10-03T21:19:12.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78802025-10-03T21:19:12.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
78812025-10-03T21:19:12.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78822025-10-03T21:19:12.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78832025-10-03T21:19:12.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78842025-10-03T21:19:12.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
78852025-10-03T21:19:12.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78862025-10-03T21:19:12.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78872025-10-03T21:19:12.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78882025-10-03T21:19:12.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78892025-10-03T21:19:12.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78902025-10-03T21:19:12.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78912025-10-03T21:19:12.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78922025-10-03T21:19:12.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
78932025-10-03T21:19:12.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78942025-10-03T21:19:12.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78952025-10-03T21:19:12.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78962025-10-03T21:19:12.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78972025-10-03T21:19:12.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78982025-10-03T21:19:12.377ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
78992025-10-03T21:19:12.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79002025-10-03T21:19:12.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79012025-10-03T21:19:12.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79022025-10-03T21:19:12.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79032025-10-03T21:19:12.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79042025-10-03T21:19:12.385ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
79052025-10-03T21:19:12.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79062025-10-03T21:19:12.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79072025-10-03T21:19:12.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79082025-10-03T21:19:12.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79092025-10-03T21:19:12.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79102025-10-03T21:19:12.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
79112025-10-03T21:19:12.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79122025-10-03T21:19:12.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79132025-10-03T21:19:12.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79142025-10-03T21:19:12.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79152025-10-03T21:19:12.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79162025-10-03T21:19:12.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
79172025-10-03T21:19:12.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79182025-10-03T21:19:12.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79192025-10-03T21:19:12.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79202025-10-03T21:19:12.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79212025-10-03T21:19:12.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79222025-10-03T21:19:12.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
79232025-10-03T21:19:12.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79242025-10-03T21:19:12.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79252025-10-03T21:19:12.403ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
79262025-10-03T21:19:12.403ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
79272025-10-03T21:19:12.403ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = d4d0feee-3973-490c-bab3-e63b3d581498 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
79282025-10-03T21:19:12.403ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
79292025-10-03T21:19:12.403ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 193.97293ms iteration = 30 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
79302025-10-03T21:19:15.194ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 146 reason = Timeout
79312025-10-03T21:19:15.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79322025-10-03T21:19:15.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79332025-10-03T21:19:15.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79342025-10-03T21:19:15.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79352025-10-03T21:19:15.199ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79362025-10-03T21:19:15.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79372025-10-03T21:19:15.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
79382025-10-03T21:19:15.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.431732ms iteration = 146 status = {"config_updated":false}
79392025-10-03T21:19:20.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
79402025-10-03T21:19:20.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79412025-10-03T21:19:20.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
79422025-10-03T21:19:20.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79432025-10-03T21:19:20.194ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 147 reason = Timeout
79442025-10-03T21:19:20.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79452025-10-03T21:19:20.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79462025-10-03T21:19:20.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79472025-10-03T21:19:20.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79482025-10-03T21:19:20.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79492025-10-03T21:19:20.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79502025-10-03T21:19:20.196ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79512025-10-03T21:19:20.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79522025-10-03T21:19:20.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79532025-10-03T21:19:20.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79542025-10-03T21:19:20.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79552025-10-03T21:19:20.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79562025-10-03T21:19:20.199ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79572025-10-03T21:19:20.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79582025-10-03T21:19:20.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79592025-10-03T21:19:20.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79602025-10-03T21:19:20.201ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
79612025-10-03T21:19:20.201ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.437214ms iteration = 147 status = {"config_updated":false}
79622025-10-03T21:19:20.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79632025-10-03T21:19:20.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
79642025-10-03T21:19:20.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79652025-10-03T21:19:20.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79662025-10-03T21:19:20.217ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
79672025-10-03T21:19:20.217ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.804588ms iteration = 74 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
79682025-10-03T21:19:20.228ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.709292ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
79692025-10-03T21:19:21.226ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 381 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = b3ea1cdd-b17f-4cf7-bf75-43930a2be7ba response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
79702025-10-03T21:19:25.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 148 reason = Timeout
79712025-10-03T21:19:25.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79722025-10-03T21:19:25.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79732025-10-03T21:19:25.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79742025-10-03T21:19:25.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79752025-10-03T21:19:25.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79762025-10-03T21:19:25.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79772025-10-03T21:19:25.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
79782025-10-03T21:19:25.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.842177ms iteration = 148 status = {"config_updated":false}
79792025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
79802025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79812025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
79822025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79832025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 149 reason = Timeout
79842025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79852025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79862025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79872025-10-03T21:19:30.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79882025-10-03T21:19:30.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79892025-10-03T21:19:30.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79902025-10-03T21:19:30.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79912025-10-03T21:19:30.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79922025-10-03T21:19:30.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79932025-10-03T21:19:30.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79942025-10-03T21:19:30.199ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79952025-10-03T21:19:30.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79962025-10-03T21:19:30.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
79972025-10-03T21:19:30.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.109561ms iteration = 149 status = {"config_updated":false}
79982025-10-03T21:19:30.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79992025-10-03T21:19:30.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
80002025-10-03T21:19:30.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80012025-10-03T21:19:30.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
80022025-10-03T21:19:30.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80032025-10-03T21:19:30.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
80042025-10-03T21:19:30.222ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
80052025-10-03T21:19:30.222ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.494903ms iteration = 75 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
80062025-10-03T21:19:30.234ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80072025-10-03T21:19:30.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80082025-10-03T21:19:30.237ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.982506ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
80092025-10-03T21:19:31.227ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 436 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 887654ff-246d-4348-84fe-834646ae5d13 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
80102025-10-03T21:19:35.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 150 reason = Timeout
80112025-10-03T21:19:35.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
80122025-10-03T21:19:35.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
80132025-10-03T21:19:35.195ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80142025-10-03T21:19:35.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
80152025-10-03T21:19:35.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
80162025-10-03T21:19:35.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
80172025-10-03T21:19:35.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
80182025-10-03T21:19:35.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.429752ms iteration = 150 status = {"config_updated":false}
80192025-10-03T21:19:36.220ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:58377
80202025-10-03T21:19:36.220ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = c9d5d68e-b138-488c-b8ef-2c4de92db056 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
80212025-10-03T21:19:36.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = c9d5d68e-b138-488c-b8ef-2c4de92db056 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
80222025-10-03T21:19:36.224ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 4034 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = c9d5d68e-b138-488c-b8ef-2c4de92db056 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending
80232025-10-03T21:19:36.224ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = 7a528e29-8f0e-4e6f-b666-dcaf2bf1be46 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
80242025-10-03T21:19:36.225ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = 7a528e29-8f0e-4e6f-b666-dcaf2bf1be46 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
80252025-10-03T21:19:36.226ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2077 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = 7a528e29-8f0e-4e6f-b666-dcaf2bf1be46 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D
80262025-10-03T21:19:40.192ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
80272025-10-03T21:19:40.192ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
80282025-10-03T21:19:40.192ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
80292025-10-03T21:19:40.192ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
80302025-10-03T21:19:40.192ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
80312025-10-03T21:19:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
80322025-10-03T21:19:40.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
80332025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
80342025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
80352025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
80362025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
80372025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80382025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
80392025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 35.931\u{b5}s iteration = 26 status = {"disabled":true,"errors":[],"sps":[]}
80402025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
80412025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
80422025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
80432025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80442025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
80452025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80462025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
80472025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 39 reason = Timeout
80482025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80492025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
80502025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80512025-10-03T21:19:40.193ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
80522025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80532025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 151 reason = Timeout
80542025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
80552025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
80562025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80572025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 41 reason = Timeout
80582025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
80592025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
80602025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
80612025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
80622025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
80632025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80642025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
80652025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
80662025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
80672025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
80682025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
80692025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
80702025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
80712025-10-03T21:19:40.193ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
80722025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
80732025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
80742025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
80752025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80762025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80772025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80782025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80792025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
80802025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
80812025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
80822025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80832025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
80842025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80852025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
80862025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
80872025-10-03T21:19:40.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
80882025-10-03T21:19:40.196ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5225, tv_nsec: 682322962 } })
80892025-10-03T21:19:40.196ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5225, tv_nsec: 682358193 } })
80902025-10-03T21:19:40.196ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.878054ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
80912025-10-03T21:19:40.196ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
80922025-10-03T21:19:40.196ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
80932025-10-03T21:19:40.196ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80942025-10-03T21:19:40.197ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
80952025-10-03T21:19:40.197ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80962025-10-03T21:19:40.197ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80972025-10-03T21:19:40.197ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
80982025-10-03T21:19:40.197ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80992025-10-03T21:19:40.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81002025-10-03T21:19:40.197ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81012025-10-03T21:19:40.197ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81022025-10-03T21:19:40.197ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.175467ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
81032025-10-03T21:19:40.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
81042025-10-03T21:19:40.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81052025-10-03T21:19:40.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81062025-10-03T21:19:40.200ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
81072025-10-03T21:19:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81082025-10-03T21:19:40.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
81092025-10-03T21:19:40.201ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81102025-10-03T21:19:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81112025-10-03T21:19:40.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
81122025-10-03T21:19:40.202ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81132025-10-03T21:19:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
81142025-10-03T21:19:40.202ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
81152025-10-03T21:19:40.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81162025-10-03T21:19:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
81172025-10-03T21:19:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
81182025-10-03T21:19:40.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81192025-10-03T21:19:40.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81202025-10-03T21:19:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
81212025-10-03T21:19:40.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81222025-10-03T21:19:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
81232025-10-03T21:19:40.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81242025-10-03T21:19:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81252025-10-03T21:19:40.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
81262025-10-03T21:19:40.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81272025-10-03T21:19:40.205ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81282025-10-03T21:19:40.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81292025-10-03T21:19:40.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81302025-10-03T21:19:40.206ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 13.150229ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
81312025-10-03T21:19:40.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
81322025-10-03T21:19:40.208ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81332025-10-03T21:19:40.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81342025-10-03T21:19:40.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81352025-10-03T21:19:40.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81362025-10-03T21:19:40.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
81372025-10-03T21:19:40.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81382025-10-03T21:19:40.213ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5225, tv_nsec: 698955830 } })
81392025-10-03T21:19:40.213ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
81402025-10-03T21:19:40.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
81412025-10-03T21:19:40.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81422025-10-03T21:19:40.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81432025-10-03T21:19:40.220ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
81442025-10-03T21:19:40.220ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 25.116423ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
81452025-10-03T21:19:40.243ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 50.180807ms iteration = 41 status = {"physical_disks_added":0}
81462025-10-03T21:19:40.268ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81472025-10-03T21:19:40.272ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
81482025-10-03T21:19:40.272ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
81492025-10-03T21:19:40.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
81502025-10-03T21:19:40.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81512025-10-03T21:19:40.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81522025-10-03T21:19:40.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
81532025-10-03T21:19:40.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
81542025-10-03T21:19:40.278ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 85.318793ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
81552025-10-03T21:19:40.279ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 85.870869ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
81562025-10-03T21:19:40.279ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 85.98744ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
81572025-10-03T21:19:40.281ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81582025-10-03T21:19:40.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
81592025-10-03T21:19:40.281ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81602025-10-03T21:19:40.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
81612025-10-03T21:19:40.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81622025-10-03T21:19:40.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
81632025-10-03T21:19:40.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 89.824244ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
81642025-10-03T21:19:40.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81652025-10-03T21:19:40.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81662025-10-03T21:19:40.286ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81672025-10-03T21:19:40.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
81682025-10-03T21:19:40.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81692025-10-03T21:19:40.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81702025-10-03T21:19:40.291ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
81712025-10-03T21:19:40.291ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
81722025-10-03T21:19:40.291ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
81732025-10-03T21:19:40.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
81742025-10-03T21:19:40.292ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 99.030658ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
81752025-10-03T21:19:40.292ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81762025-10-03T21:19:40.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
81772025-10-03T21:19:40.293ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 100.09799ms iteration = 26 status = 0
81782025-10-03T21:19:40.295ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81792025-10-03T21:19:40.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
81802025-10-03T21:19:40.297ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81812025-10-03T21:19:40.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81822025-10-03T21:19:40.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
81832025-10-03T21:19:40.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
81842025-10-03T21:19:40.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81852025-10-03T21:19:40.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
81862025-10-03T21:19:40.302ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81872025-10-03T21:19:40.304ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81882025-10-03T21:19:40.305ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
81892025-10-03T21:19:40.306ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81902025-10-03T21:19:40.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
81912025-10-03T21:19:40.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
81922025-10-03T21:19:40.308ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
81932025-10-03T21:19:40.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
81942025-10-03T21:19:40.311ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
81952025-10-03T21:19:40.311ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 118.012572ms iteration = 151 status = {"config_updated":false}
81962025-10-03T21:19:40.323ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
81972025-10-03T21:19:40.323ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 129.856256ms iteration = 76 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
81982025-10-03T21:19:40.324ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
81992025-10-03T21:19:40.324ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82002025-10-03T21:19:40.325ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ce2cc2d-aebe-4678-a406-05d024360635", "content-length": "26", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82012025-10-03T21:19:40.325ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82022025-10-03T21:19:40.327ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
82032025-10-03T21:19:40.330ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
82042025-10-03T21:19:40.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
82052025-10-03T21:19:40.331ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82062025-10-03T21:19:40.331ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
82072025-10-03T21:19:40.331ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 138.19852ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
82082025-10-03T21:19:40.332ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82092025-10-03T21:19:40.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
82102025-10-03T21:19:40.335ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
82112025-10-03T21:19:40.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
82122025-10-03T21:19:40.337ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
82132025-10-03T21:19:40.337ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 143.812114ms iteration = 39 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
82142025-10-03T21:19:40.350ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
82152025-10-03T21:19:40.350ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82162025-10-03T21:19:40.350ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c56d8c04-a823-47c4-a73b-ce76befb63cb", "content-length": "26", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82172025-10-03T21:19:40.350ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82182025-10-03T21:19:40.351ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
82192025-10-03T21:19:40.352ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82202025-10-03T21:19:40.352ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
82212025-10-03T21:19:40.352ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9afdb20-240a-493d-ad95-8e8f8e3e8e75", "content-length": "26", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82222025-10-03T21:19:40.352ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82232025-10-03T21:19:40.354ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
82242025-10-03T21:19:40.391ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
82252025-10-03T21:19:40.391ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
82262025-10-03T21:19:40.391ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "21119bf5-76c1-478f-ac34-590121729f29", "content-length": "372", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82272025-10-03T21:19:40.391ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82282025-10-03T21:19:40.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82292025-10-03T21:19:40.408ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
82302025-10-03T21:19:40.408ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
82312025-10-03T21:19:40.409ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "32bc9787-3b26-4af7-9fae-b6295c4f49d1", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82322025-10-03T21:19:40.409ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82332025-10-03T21:19:40.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82342025-10-03T21:19:40.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82352025-10-03T21:19:40.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82362025-10-03T21:19:40.415ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82372025-10-03T21:19:40.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82382025-10-03T21:19:40.422ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
82392025-10-03T21:19:40.422ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
82402025-10-03T21:19:40.422ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e4463fba-e02a-4067-9a4f-1b6301deec97", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82412025-10-03T21:19:40.424ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 230.743865ms iteration = 26 status = {}
82422025-10-03T21:19:40.471ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
82432025-10-03T21:19:40.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
82442025-10-03T21:19:40.473ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 280.911272ms iteration = 26 status = null
82452025-10-03T21:19:40.475ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
82462025-10-03T21:19:40.476ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "10785b7d-2c62-4b50-897e-f398f0884f9c", "content-length": "691", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82472025-10-03T21:19:40.476ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
82482025-10-03T21:19:40.476ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
82492025-10-03T21:19:40.476ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
82502025-10-03T21:19:40.476ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 284.117778ms iteration = 26 status = {}
82512025-10-03T21:19:40.477ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 284.167938ms iteration = 26 status = {}
82522025-10-03T21:19:40.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82532025-10-03T21:19:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82542025-10-03T21:19:40.540ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
82552025-10-03T21:19:40.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82562025-10-03T21:19:40.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82572025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
82582025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
82592025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
82602025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
82612025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82622025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82632025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82642025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
82652025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82662025-10-03T21:19:40.552ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
82672025-10-03T21:19:40.553ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
82682025-10-03T21:19:40.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82692025-10-03T21:19:40.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82702025-10-03T21:19:40.555ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
82712025-10-03T21:19:40.555ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
82722025-10-03T21:19:40.811ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f0526824-e153-4a5c-94bc-9b2cdb9cc258", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82732025-10-03T21:19:40.811ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
82742025-10-03T21:19:40.811ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
82752025-10-03T21:19:40.811ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b75d4a21-f179-479c-b207-a3a93aadaa61", "content-length": "682", "date": "Fri, 03 Oct 2025 21:19:40 GMT"} })
82762025-10-03T21:19:40.811ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82772025-10-03T21:19:40.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82782025-10-03T21:19:40.815ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82792025-10-03T21:19:40.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82802025-10-03T21:19:40.820ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82812025-10-03T21:19:40.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82822025-10-03T21:19:40.822ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82832025-10-03T21:19:40.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82842025-10-03T21:19:40.830ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 636.860008ms iteration = 26 status = {}
82852025-10-03T21:19:41.227ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 598 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 1a4f9dcd-364e-4a58-b15d-9c2b64d3afec response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
82862025-10-03T21:19:45.194ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 152 reason = Timeout
82872025-10-03T21:19:45.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82882025-10-03T21:19:45.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82892025-10-03T21:19:45.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82902025-10-03T21:19:45.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
82912025-10-03T21:19:45.199ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82922025-10-03T21:19:45.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82932025-10-03T21:19:45.201ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
82942025-10-03T21:19:45.201ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.514294ms iteration = 152 status = {"config_updated":false}
82952025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
82962025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82972025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
82982025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 153 reason = Timeout
82992025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83002025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83012025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83022025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83032025-10-03T21:19:50.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83042025-10-03T21:19:50.196ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83052025-10-03T21:19:50.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83062025-10-03T21:19:50.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83072025-10-03T21:19:50.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83082025-10-03T21:19:50.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
83092025-10-03T21:19:50.200ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83102025-10-03T21:19:50.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83112025-10-03T21:19:50.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
83122025-10-03T21:19:50.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83132025-10-03T21:19:50.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83142025-10-03T21:19:50.201ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
83152025-10-03T21:19:50.201ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.486014ms iteration = 153 status = {"config_updated":false}
83162025-10-03T21:19:50.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83172025-10-03T21:19:50.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83182025-10-03T21:19:50.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83192025-10-03T21:19:50.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
83202025-10-03T21:19:50.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83212025-10-03T21:19:50.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83222025-10-03T21:19:50.220ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
83232025-10-03T21:19:50.220ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.919372ms iteration = 77 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
83242025-10-03T21:19:50.234ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.294954ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
83252025-10-03T21:19:51.227ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 403 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = f3e40465-c94b-4a00-8a64-b53b47aebc46 response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
83262025-10-03T21:19:55.193ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 154 reason = Timeout
83272025-10-03T21:19:55.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83282025-10-03T21:19:55.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83292025-10-03T21:19:55.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83302025-10-03T21:19:55.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
83312025-10-03T21:19:55.198ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83322025-10-03T21:19:55.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83332025-10-03T21:19:55.200ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
83342025-10-03T21:19:55.200ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.765136ms iteration = 154 status = {"config_updated":false}
83352025-10-03T21:20:00.194ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
83362025-10-03T21:20:00.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83372025-10-03T21:20:00.194ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
83382025-10-03T21:20:00.194ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83392025-10-03T21:20:00.194ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 155 reason = Timeout
83402025-10-03T21:20:00.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83412025-10-03T21:20:00.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83422025-10-03T21:20:00.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83432025-10-03T21:20:00.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83442025-10-03T21:20:00.196ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83452025-10-03T21:20:00.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83462025-10-03T21:20:00.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83472025-10-03T21:20:00.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83482025-10-03T21:20:00.198ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.58605ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
83492025-10-03T21:20:00.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
83502025-10-03T21:20:00.200ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83512025-10-03T21:20:00.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
83522025-10-03T21:20:00.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83532025-10-03T21:20:00.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83542025-10-03T21:20:00.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83552025-10-03T21:20:00.201ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
83562025-10-03T21:20:00.201ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.09148ms iteration = 155 status = {"config_updated":false}
83572025-10-03T21:20:00.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83582025-10-03T21:20:00.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83592025-10-03T21:20:00.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83602025-10-03T21:20:00.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
83612025-10-03T21:20:00.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83622025-10-03T21:20:00.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83632025-10-03T21:20:00.216ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
83642025-10-03T21:20:00.216ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.948908ms iteration = 78 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
83652025-10-03T21:20:01.228ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 734 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = ae889a1c-4646-4043-a087-d5eded27bead response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
83662025-10-03T21:20:05.194ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 156 reason = Timeout
83672025-10-03T21:20:05.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83682025-10-03T21:20:05.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83692025-10-03T21:20:05.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83702025-10-03T21:20:05.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
83712025-10-03T21:20:05.199ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83722025-10-03T21:20:05.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83732025-10-03T21:20:05.201ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
83742025-10-03T21:20:05.201ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.139211ms iteration = 156 status = {"config_updated":false}
83752025-10-03T21:20:06.220ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = 1e4ce9da-20f5-4a76-90ae-525d984cd5cc roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
83762025-10-03T21:20:06.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = 1e4ce9da-20f5-4a76-90ae-525d984cd5cc resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
83772025-10-03T21:20:06.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3084 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = 1e4ce9da-20f5-4a76-90ae-525d984cd5cc response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?sort_by=id_ascending
83782025-10-03T21:20:06.223ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = e1ef587c-1300-4de8-a039-f52e316e52e3 roles = RoleSet { roles: {} } uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
83792025-10-03T21:20:06.224ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = e1ef587c-1300-4de8-a039-f52e316e52e3 resource = Database result = Ok(()) uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
83802025-10-03T21:20:06.226ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2928 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58377 req_id = e1ef587c-1300-4de8-a039-f52e316e52e3 response_code = 200 uri = /metrics/collectors/d03f63a5-05d6-4fcc-b79e-7d63884498be/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMmU1NWVhNC1mNWFmLTRhMGYtYWNhMy0wYjUxNTE4MzJlNDgifX0%3D
83812025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
83822025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
83832025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
83842025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
83852025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
83862025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
83872025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
83882025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
83892025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
83902025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
83912025-10-03T21:20:10.193ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
83922025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 27 reason = Timeout
83932025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
83942025-10-03T21:20:10.193ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = ExternalNetworking step id = Ensure
83952025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
83962025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83972025-10-03T21:20:10.193ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-10-03 21:10:10.193373325 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
83982025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
83992025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
84002025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
84012025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
84022025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
84032025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
84042025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
84052025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
84062025-10-03T21:20:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
84072025-10-03T21:20:10.193ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("486849a9-1733-4aa9-9b66-43ace2da0eea.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("6436aa1b-d53b-419a-817f-0a80ade8ca28.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 5255, tv_nsec: 679586071 } })
84082025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
84092025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
84102025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
84112025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
84122025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
84132025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 40 reason = Timeout
84142025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84152025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
84162025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
84172025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
84182025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
84192025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
84202025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
84212025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
84222025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
84232025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
84242025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
84252025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
84262025-10-03T21:20:10.194ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2efcfb05-0c3d-45aa-b120-c334f3706be4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9a7c1656-d513-4cc3-b2b0-8e31ad2b5545.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("cac0d292-f20b-4f3a-b918-a96424ce10fa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 5255, tv_nsec: 679801374 } })
84272025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
84282025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 31 reason = Timeout
84292025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84302025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
84312025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
84322025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
84332025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
84342025-10-03T21:20:10.194ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
84352025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
84362025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
84372025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84382025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
84392025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
84402025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
84412025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
84422025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84432025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
84442025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
84452025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
84462025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
84472025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
84482025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84492025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
84502025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 157 reason = Timeout
84512025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
84522025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
84532025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
84542025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
84552025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
84562025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
84572025-10-03T21:20:10.194ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
84582025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
84592025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
84602025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
84612025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 27 reason = Timeout
84622025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.741\u{b5}s iteration = 27 status = {"disabled":true,"errors":[],"sps":[]}
84632025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
84642025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
84652025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
84662025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
84672025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 42 reason = Timeout
84682025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
84692025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84702025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
84712025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84722025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
84732025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
84742025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
84752025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84762025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
84772025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
84782025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84792025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
84802025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
84812025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
84822025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
84832025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
84842025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
84852025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
84862025-10-03T21:20:10.194ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.775ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
84872025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
84882025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
84892025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
84902025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
84912025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.73393ms iteration = 14 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
84922025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
84932025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
84942025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
84952025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
84962025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
84972025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
84982025-10-03T21:20:10.195ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
84992025-10-03T21:20:10.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
85002025-10-03T21:20:10.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
85012025-10-03T21:20:10.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
85022025-10-03T21:20:10.198ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5255, tv_nsec: 684509967 } })
85032025-10-03T21:20:10.199ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1267
85042025-10-03T21:20:10.199ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5255, tv_nsec: 685337146 } })
85052025-10-03T21:20:10.200ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1267
85062025-10-03T21:20:10.201ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85072025-10-03T21:20:10.201ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
85082025-10-03T21:20:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
85092025-10-03T21:20:10.202ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
85102025-10-03T21:20:10.202ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 8.293604ms iteration = 14 status = {"by_rx":{},"error":null}
85112025-10-03T21:20:10.202ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-10-03 21:10:10.193373325 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
85122025-10-03T21:20:10.203ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.635898ms iteration = 14 status = {"expiration":"2025-10-03T21:10:10.193373325Z","pruned":[]}
85132025-10-03T21:20:10.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85142025-10-03T21:20:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
85152025-10-03T21:20:10.204ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
85162025-10-03T21:20:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
85172025-10-03T21:20:10.207ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85182025-10-03T21:20:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
85192025-10-03T21:20:10.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85202025-10-03T21:20:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
85212025-10-03T21:20:10.213ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 20.270728ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
85222025-10-03T21:20:10.213ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85232025-10-03T21:20:10.214ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85242025-10-03T21:20:10.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
85252025-10-03T21:20:10.215ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85262025-10-03T21:20:10.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85272025-10-03T21:20:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
85282025-10-03T21:20:10.217ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
85292025-10-03T21:20:10.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
85302025-10-03T21:20:10.221ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85312025-10-03T21:20:10.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
85322025-10-03T21:20:10.221ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
85332025-10-03T21:20:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
85342025-10-03T21:20:10.223ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 29.728936ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
85352025-10-03T21:20:10.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
85362025-10-03T21:20:10.236ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85372025-10-03T21:20:10.240ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
85382025-10-03T21:20:10.240ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85392025-10-03T21:20:10.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85402025-10-03T21:20:10.252ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
85412025-10-03T21:20:10.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
85422025-10-03T21:20:10.260ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85432025-10-03T21:20:10.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
85442025-10-03T21:20:10.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
85452025-10-03T21:20:10.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
85462025-10-03T21:20:10.270ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
85472025-10-03T21:20:10.270ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
85482025-10-03T21:20:10.270ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
85492025-10-03T21:20:10.270ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
85502025-10-03T21:20:10.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
85512025-10-03T21:20:10.273ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 79.125903ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
85522025-10-03T21:20:10.273ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
85532025-10-03T21:20:10.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
85542025-10-03T21:20:10.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85552025-10-03T21:20:10.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85562025-10-03T21:20:10.285ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85572025-10-03T21:20:10.288ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
85582025-10-03T21:20:10.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
85592025-10-03T21:20:10.289ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85602025-10-03T21:20:10.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
85612025-10-03T21:20:10.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85622025-10-03T21:20:10.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85632025-10-03T21:20:10.291ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85642025-10-03T21:20:10.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
85652025-10-03T21:20:10.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
85662025-10-03T21:20:10.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
85672025-10-03T21:20:10.294ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
85682025-10-03T21:20:10.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85692025-10-03T21:20:10.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85702025-10-03T21:20:10.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
85712025-10-03T21:20:10.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85722025-10-03T21:20:10.301ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
85732025-10-03T21:20:10.301ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
85742025-10-03T21:20:10.301ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
85752025-10-03T21:20:10.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
85762025-10-03T21:20:10.302ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85772025-10-03T21:20:10.302ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85782025-10-03T21:20:10.303ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 109.241103ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"195.0.0","globs_reprocessed":{},"no_receivers":[]}
85792025-10-03T21:20:10.305ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85802025-10-03T21:20:10.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
85812025-10-03T21:20:10.306ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
85822025-10-03T21:20:10.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
85832025-10-03T21:20:10.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85842025-10-03T21:20:10.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
85852025-10-03T21:20:10.308ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
85862025-10-03T21:20:10.308ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
85872025-10-03T21:20:10.308ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
85882025-10-03T21:20:10.308ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 114.020878ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
85892025-10-03T21:20:10.308ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5255, tv_nsec: 794173475 } })
85902025-10-03T21:20:10.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
85912025-10-03T21:20:10.308ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1267 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
85922025-10-03T21:20:10.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
85932025-10-03T21:20:10.311ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
85942025-10-03T21:20:10.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
85952025-10-03T21:20:10.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675bea75-8e2d-4986-86f2-b590a30bb084, lookup_type: ById(675bea75-8e2d-4986-86f2-b590a30bb084) } result = Ok(())
85962025-10-03T21:20:10.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85972025-10-03T21:20:10.312ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
85982025-10-03T21:20:10.312ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
85992025-10-03T21:20:10.312ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
86002025-10-03T21:20:10.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
86012025-10-03T21:20:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
86022025-10-03T21:20:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
86032025-10-03T21:20:10.323ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
86042025-10-03T21:20:10.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
86052025-10-03T21:20:10.331ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 137.444372ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
86062025-10-03T21:20:10.334ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
86072025-10-03T21:20:10.334ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
86082025-10-03T21:20:10.334ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
86092025-10-03T21:20:10.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
86102025-10-03T21:20:10.344ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
86112025-10-03T21:20:10.344ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 149.650149ms iteration = 157 status = {"config_updated":false}
86122025-10-03T21:20:10.345ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 150.718161ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
86132025-10-03T21:20:10.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86142025-10-03T21:20:10.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86152025-10-03T21:20:10.349ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
86162025-10-03T21:20:10.349ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
86172025-10-03T21:20:10.349ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee304f38-b873-436e-8caa-e7fe27998f83", "content-length": "26", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
86182025-10-03T21:20:10.350ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
86192025-10-03T21:20:10.352ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86202025-10-03T21:20:10.357ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
86212025-10-03T21:20:10.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
86222025-10-03T21:20:10.359ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
86232025-10-03T21:20:10.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
86242025-10-03T21:20:10.364ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86252025-10-03T21:20:10.365ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 171.90344ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
86262025-10-03T21:20:10.367ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
86272025-10-03T21:20:10.367ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
86282025-10-03T21:20:10.367ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 172.657269ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
86292025-10-03T21:20:10.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86302025-10-03T21:20:10.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
86312025-10-03T21:20:10.368ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
86322025-10-03T21:20:10.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
86332025-10-03T21:20:10.369ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 175.558562ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
86342025-10-03T21:20:10.370ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 original_time_created = 2025-10-03 21:06:58.531485 UTC target_id = 675bea75-8e2d-4986-86f2-b590a30bb084 time_created = 2025-10-03 21:06:58.531485 UTC
86352025-10-03T21:20:10.370ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 176.25215ms iteration = 79 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"675bea75-8e2d-4986-86f2-b590a30bb084","time_created":"2025-10-03T21:06:58.531485Z"}
86362025-10-03T21:20:10.372ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
86372025-10-03T21:20:10.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86382025-10-03T21:20:10.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86392025-10-03T21:20:10.372ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
86402025-10-03T21:20:10.372ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 178.538546ms iteration = 40 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
86412025-10-03T21:20:10.374ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86422025-10-03T21:20:10.377ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
86432025-10-03T21:20:10.377ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
86442025-10-03T21:20:10.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
86452025-10-03T21:20:10.379ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86462025-10-03T21:20:10.383ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
86472025-10-03T21:20:10.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
86482025-10-03T21:20:10.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86492025-10-03T21:20:10.385ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
86502025-10-03T21:20:10.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86512025-10-03T21:20:10.385ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
86522025-10-03T21:20:10.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
86532025-10-03T21:20:10.388ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
86542025-10-03T21:20:10.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86552025-10-03T21:20:10.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86562025-10-03T21:20:10.390ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86572025-10-03T21:20:10.392ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86582025-10-03T21:20:10.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
86592025-10-03T21:20:10.396ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
86602025-10-03T21:20:10.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
86612025-10-03T21:20:10.396ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
86622025-10-03T21:20:10.396ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
86632025-10-03T21:20:10.396ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
86642025-10-03T21:20:10.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
86652025-10-03T21:20:10.402ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 208.372273ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
86662025-10-03T21:20:10.404ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
86672025-10-03T21:20:10.404ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86682025-10-03T21:20:10.409ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 77c81759-27f5-47ab-a3af-9094ce6a8b01 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 238731e0-874e-47e4-a037-a86612882fbd zone_kind = nexus
86692025-10-03T21:20:10.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
86702025-10-03T21:20:10.409ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
86712025-10-03T21:20:10.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
86722025-10-03T21:20:10.415ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
86732025-10-03T21:20:10.415ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
86742025-10-03T21:20:10.415ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-10-03 21:07:12.445842 UTC dns_group = internal generation = 1
86752025-10-03T21:20:10.415ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 222.175998ms iteration = 14 status = {"generation":1}
86762025-10-03T21:20:10.421ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
86772025-10-03T21:20:10.424ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 00b499c6-b7dd-4b87-970b-f668f6348d3d (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 486849a9-1733-4aa9-9b66-43ace2da0eea zone_kind = external_dns
86782025-10-03T21:20:10.426ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
86792025-10-03T21:20:10.428ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
86802025-10-03T21:20:10.428ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
86812025-10-03T21:20:10.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
86822025-10-03T21:20:10.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
86832025-10-03T21:20:10.429ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86842025-10-03T21:20:10.430ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 3da63806-02ea-4173-a1d0-1a220c08646f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6436aa1b-d53b-419a-817f-0a80ade8ca28 zone_kind = external_dns
86852025-10-03T21:20:10.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86862025-10-03T21:20:10.433ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
86872025-10-03T21:20:10.436ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86882025-10-03T21:20:10.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86892025-10-03T21:20:10.436ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: 116e0cfb-2214-476c-9d95-64a72b4d418c (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a5fae325-6f04-4795-bc31-fbf69228d92b zone_kind = nexus
86902025-10-03T21:20:10.438ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1275 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c zone_address = fd00:1122:3344:101::2
86912025-10-03T21:20:10.438ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
86922025-10-03T21:20:10.439ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9618f45d-e113-4a1b-9271-76353af88318", "content-length": "26", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
86932025-10-03T21:20:10.439ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1278 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
86942025-10-03T21:20:10.439ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
86952025-10-03T21:20:10.441ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
86962025-10-03T21:20:10.442ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Floating(OmicronZoneExternalFloatingIp { id: e382f5c4-6008-44b5-8f2e-a858cb2aa687 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = a969a278-39b1-4a1a-852e-8c04afad82f6 zone_kind = nexus
86972025-10-03T21:20:10.444ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:209 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
86982025-10-03T21:20:10.446ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
86992025-10-03T21:20:10.446ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 252.103566ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
87002025-10-03T21:20:10.447ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:257 ip = Snat(OmicronZoneExternalSnatIp { id: f523b636-249d-4439-8476-b9a89398b997 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: eaa94b35-a3ef-42c3-a207-3dff0340936f, kind: Service { id: f8952d36-0c9f-4327-9fce-f8d697d4e626 }, name: Name("ntp-f8952d36-0c9f-4327-9fce-f8d697d4e626"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 12])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 zone_kind = boundary_ntp
87012025-10-03T21:20:10.447ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = SledList step id = Fetch
87022025-10-03T21:20:10.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87032025-10-03T21:20:10.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87042025-10-03T21:20:10.448ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
87052025-10-03T21:20:10.448ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
87062025-10-03T21:20:10.448ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
87072025-10-03T21:20:10.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
87082025-10-03T21:20:10.454ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
87092025-10-03T21:20:10.457ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
87102025-10-03T21:20:10.467ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1275 zone_address = fd00:1122:3344:101::2
87112025-10-03T21:20:10.467ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
87122025-10-03T21:20:10.468ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1be1290a-54cf-409b-9987-f84bf28e0af6", "content-length": "26", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
87132025-10-03T21:20:10.468ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1278 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
87142025-10-03T21:20:10.469ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1312 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
87152025-10-03T21:20:10.473ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87162025-10-03T21:20:10.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
87172025-10-03T21:20:10.481ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87182025-10-03T21:20:10.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
87192025-10-03T21:20:10.489ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87202025-10-03T21:20:10.489ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
87212025-10-03T21:20:10.489ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
87222025-10-03T21:20:10.489ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 295.077571ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
87232025-10-03T21:20:10.489ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87242025-10-03T21:20:10.489ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87252025-10-03T21:20:10.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87262025-10-03T21:20:10.489ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87272025-10-03T21:20:10.489ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 295.352374ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
87282025-10-03T21:20:10.490ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 295.736908ms iteration = 42 status = {"physical_disks_added":0}
87292025-10-03T21:20:10.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
87302025-10-03T21:20:10.493ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
87312025-10-03T21:20:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
87322025-10-03T21:20:10.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
87332025-10-03T21:20:10.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87342025-10-03T21:20:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87352025-10-03T21:20:10.494ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
87362025-10-03T21:20:10.494ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87372025-10-03T21:20:10.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
87382025-10-03T21:20:10.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
87392025-10-03T21:20:10.495ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
87402025-10-03T21:20:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
87412025-10-03T21:20:10.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
87422025-10-03T21:20:10.496ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
87432025-10-03T21:20:10.498ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87442025-10-03T21:20:10.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
87452025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87462025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
87472025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87482025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 307.521151ms iteration = 27 status = 0
87492025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
87502025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
87512025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
87522025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-10-03 21:07:14.470425 UTC dns_group = external generation = 2
87532025-10-03T21:20:10.501ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 308.510983ms iteration = 14 status = {"generation":2}
87542025-10-03T21:20:10.503ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
87552025-10-03T21:20:10.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87562025-10-03T21:20:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87572025-10-03T21:20:10.505ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 312.189155ms iteration = 27 status = {}
87582025-10-03T21:20:10.549ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
87592025-10-03T21:20:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
87602025-10-03T21:20:10.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87612025-10-03T21:20:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87622025-10-03T21:20:10.569ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-10-03T21:07:04.185535Z, time_modified: 2025-10-03T21:07:04.185535Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, db_silo: Silo { identity: SiloIdentity { id: 561d1a2f-ded1-4ef8-bc80-3f602ddebca1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-10-03T21:07:14.410660Z, time_modified: 2025-10-03T21:07:14.410660Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030" }] }) }
87632025-10-03T21:20:10.569ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 376.292207ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]}},"default_endpoint":{"silo_id":"561d1a2f-ded1-4ef8-bc80-3f602ddebca1","tls_certs":["45429f7fe1e6f9be84d8bbceaabdc9c053aa7f1fb2ef6207fb3a47ab4471e030"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
87642025-10-03T21:20:10.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87652025-10-03T21:20:10.611ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
87662025-10-03T21:20:10.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87672025-10-03T21:20:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87682025-10-03T21:20:10.613ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = DeployNexusRecords step id = Ensure
87692025-10-03T21:20:10.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87702025-10-03T21:20:10.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87712025-10-03T21:20:10.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87722025-10-03T21:20:10.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87732025-10-03T21:20:10.616ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
87742025-10-03T21:20:10.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87752025-10-03T21:20:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87762025-10-03T21:20:10.617ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
87772025-10-03T21:20:10.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87782025-10-03T21:20:10.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87792025-10-03T21:20:10.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87802025-10-03T21:20:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87812025-10-03T21:20:10.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87822025-10-03T21:20:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
87832025-10-03T21:20:10.626ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
87842025-10-03T21:20:10.626ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "6f831008-a095-4639-ac71-3a2506e06193", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87852025-10-03T21:20:10.626ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87862025-10-03T21:20:10.628ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "549391a1-0a80-4945-ae94-491a7f508555", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87872025-10-03T21:20:10.628ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87882025-10-03T21:20:10.628ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
87892025-10-03T21:20:10.630ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87902025-10-03T21:20:10.630ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 437.140644ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
87912025-10-03T21:20:10.632ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
87922025-10-03T21:20:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
87932025-10-03T21:20:10.635ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 442.12368ms iteration = 27 status = null
87942025-10-03T21:20:10.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87952025-10-03T21:20:10.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87962025-10-03T21:20:10.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87972025-10-03T21:20:10.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87982025-10-03T21:20:10.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87992025-10-03T21:20:10.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
88002025-10-03T21:20:10.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88012025-10-03T21:20:10.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88022025-10-03T21:20:10.667ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = SledAgent step id = Ensure
88032025-10-03T21:20:10.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88042025-10-03T21:20:10.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88052025-10-03T21:20:10.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88062025-10-03T21:20:10.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
88072025-10-03T21:20:10.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88082025-10-03T21:20:10.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88092025-10-03T21:20:10.675ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
88102025-10-03T21:20:10.675ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
88112025-10-03T21:20:10.675ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
88122025-10-03T21:20:10.675ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe964d04-ebc0-4d73-98ec-75703d85c6a4", "content-length": "372", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
88132025-10-03T21:20:10.675ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88142025-10-03T21:20:10.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88152025-10-03T21:20:10.676ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "693bfafe-e408-45e5-8e00-770a34866b9d", "content-length": "691", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
88162025-10-03T21:20:10.676ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
88172025-10-03T21:20:10.676ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
88182025-10-03T21:20:10.676ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
88192025-10-03T21:20:10.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88202025-10-03T21:20:10.677ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 483.306755ms iteration = 27 status = {}
88212025-10-03T21:20:10.679ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
88222025-10-03T21:20:10.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88232025-10-03T21:20:10.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88242025-10-03T21:20:10.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88252025-10-03T21:20:10.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88262025-10-03T21:20:10.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88272025-10-03T21:20:10.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
88282025-10-03T21:20:10.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88292025-10-03T21:20:10.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88302025-10-03T21:20:10.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88312025-10-03T21:20:10.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88322025-10-03T21:20:10.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88332025-10-03T21:20:10.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88342025-10-03T21:20:10.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88352025-10-03T21:20:10.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
88362025-10-03T21:20:10.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88372025-10-03T21:20:10.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88382025-10-03T21:20:10.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88392025-10-03T21:20:10.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
88402025-10-03T21:20:10.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88412025-10-03T21:20:10.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88422025-10-03T21:20:10.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88432025-10-03T21:20:10.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88442025-10-03T21:20:10.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88452025-10-03T21:20:10.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88462025-10-03T21:20:10.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88472025-10-03T21:20:10.708ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
88482025-10-03T21:20:10.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88492025-10-03T21:20:10.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88502025-10-03T21:20:10.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88512025-10-03T21:20:10.712ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
88522025-10-03T21:20:10.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88532025-10-03T21:20:10.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88542025-10-03T21:20:10.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88552025-10-03T21:20:10.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88562025-10-03T21:20:10.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88572025-10-03T21:20:10.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88582025-10-03T21:20:10.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88592025-10-03T21:20:10.719ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
88602025-10-03T21:20:10.719ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 524.919035ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
88612025-10-03T21:20:10.720ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
88622025-10-03T21:20:10.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88632025-10-03T21:20:10.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88642025-10-03T21:20:10.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88652025-10-03T21:20:10.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88662025-10-03T21:20:10.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88672025-10-03T21:20:10.726ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
88682025-10-03T21:20:10.726ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
88692025-10-03T21:20:10.726ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7d970e32-9c35-486c-a5f0-5157e38a79b1", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
88702025-10-03T21:20:10.727ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88712025-10-03T21:20:10.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88722025-10-03T21:20:10.728ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
88732025-10-03T21:20:10.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88742025-10-03T21:20:10.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88752025-10-03T21:20:10.729ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88762025-10-03T21:20:10.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88772025-10-03T21:20:10.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88782025-10-03T21:20:10.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88792025-10-03T21:20:10.733ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88802025-10-03T21:20:10.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88812025-10-03T21:20:10.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88822025-10-03T21:20:10.736ZINFOnexus (BackgroundTasks): VPC route manager sled 87c00105-6187-43fb-ab98-0bf5dda28bd9 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
88832025-10-03T21:20:10.736ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
88842025-10-03T21:20:10.737ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a0efcc43-6926-4ae6-926e-802ed9d447af", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
88852025-10-03T21:20:10.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
88862025-10-03T21:20:10.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88872025-10-03T21:20:10.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88882025-10-03T21:20:10.739ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 545.409296ms iteration = 27 status = {}
88892025-10-03T21:20:10.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88902025-10-03T21:20:10.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88912025-10-03T21:20:10.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88922025-10-03T21:20:10.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
88932025-10-03T21:20:10.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88942025-10-03T21:20:10.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
88952025-10-03T21:20:10.748ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
88962025-10-03T21:20:10.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88972025-10-03T21:20:10.749ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "ce4a38dc-2760-4022-98a6-1c2cd1c1c1ef", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88982025-10-03T21:20:10.749ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88992025-10-03T21:20:10.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
89002025-10-03T21:20:10.751ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "4f4be01a-98a6-4f94-852c-929f21043424", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
89012025-10-03T21:20:10.751ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
89022025-10-03T21:20:10.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89032025-10-03T21:20:10.753ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "02ee516f-c785-4d50-b865-c3218cfde566", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
89042025-10-03T21:20:10.753ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
89052025-10-03T21:20:10.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
89062025-10-03T21:20:10.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
89072025-10-03T21:20:10.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
89082025-10-03T21:20:10.758ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
89092025-10-03T21:20:10.758ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 565.404672ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
89102025-10-03T21:20:10.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
89112025-10-03T21:20:10.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
89122025-10-03T21:20:10.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89132025-10-03T21:20:10.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
89142025-10-03T21:20:10.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
89152025-10-03T21:20:10.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
89162025-10-03T21:20:10.764ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
89172025-10-03T21:20:10.765ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
89182025-10-03T21:20:10.765ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
89192025-10-03T21:20:10.765ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 360fda21-50ab-40a4-9369-3a541bf9d3e4 parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
89202025-10-03T21:20:10.765ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
89212025-10-03T21:20:10.765ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 571.527222ms iteration = 31 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
89222025-10-03T21:20:10.766ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "c07e914d-260d-4c59-8d94-279af531b156", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} }) sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
89232025-10-03T21:20:10.768ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = FirewallRules step id = Ensure
89242025-10-03T21:20:10.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89252025-10-03T21:20:10.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89262025-10-03T21:20:10.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89272025-10-03T21:20:10.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89282025-10-03T21:20:10.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89292025-10-03T21:20:10.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89302025-10-03T21:20:10.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89312025-10-03T21:20:10.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89322025-10-03T21:20:10.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89332025-10-03T21:20:10.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89342025-10-03T21:20:10.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89352025-10-03T21:20:10.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89362025-10-03T21:20:10.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89372025-10-03T21:20:10.781ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89382025-10-03T21:20:10.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89392025-10-03T21:20:10.783ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
89402025-10-03T21:20:10.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89412025-10-03T21:20:10.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89422025-10-03T21:20:10.784ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 88b6071e-6a35-46e2-b1e0-598bb248e92b, rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(4e41434f-420c-439d-a7d8-e8be849940e3) } rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
89432025-10-03T21:20:10.784ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89442025-10-03T21:20:10.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89452025-10-03T21:20:10.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89462025-10-03T21:20:10.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89472025-10-03T21:20:10.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89482025-10-03T21:20:10.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89492025-10-03T21:20:10.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89502025-10-03T21:20:10.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
89512025-10-03T21:20:10.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89522025-10-03T21:20:10.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89532025-10-03T21:20:10.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89542025-10-03T21:20:10.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89552025-10-03T21:20:10.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89562025-10-03T21:20:10.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89572025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
89582025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 4e41434f-420c-439d-a7d8-e8be849940e3
89592025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
89602025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
89612025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
89622025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
89632025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
89642025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
89652025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
89662025-10-03T21:20:10.797ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
89672025-10-03T21:20:10.798ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c
89682025-10-03T21:20:10.798ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89692025-10-03T21:20:10.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89702025-10-03T21:20:10.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89712025-10-03T21:20:10.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89722025-10-03T21:20:10.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89732025-10-03T21:20:10.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89742025-10-03T21:20:10.800ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch_location = Switch0
89752025-10-03T21:20:10.800ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
89762025-10-03T21:20:10.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89772025-10-03T21:20:10.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89782025-10-03T21:20:10.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89792025-10-03T21:20:10.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89802025-10-03T21:20:10.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89812025-10-03T21:20:10.807ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
89822025-10-03T21:20:10.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89832025-10-03T21:20:10.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89842025-10-03T21:20:10.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89852025-10-03T21:20:10.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89862025-10-03T21:20:10.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89872025-10-03T21:20:10.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89882025-10-03T21:20:10.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89892025-10-03T21:20:10.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89902025-10-03T21:20:10.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89912025-10-03T21:20:10.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89922025-10-03T21:20:10.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89932025-10-03T21:20:10.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89942025-10-03T21:20:10.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89952025-10-03T21:20:10.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89962025-10-03T21:20:10.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89972025-10-03T21:20:10.823ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
89982025-10-03T21:20:10.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89992025-10-03T21:20:10.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90002025-10-03T21:20:10.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90012025-10-03T21:20:10.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90022025-10-03T21:20:10.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90032025-10-03T21:20:10.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90042025-10-03T21:20:10.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90052025-10-03T21:20:10.831ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
90062025-10-03T21:20:10.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90072025-10-03T21:20:10.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90082025-10-03T21:20:10.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90092025-10-03T21:20:10.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90102025-10-03T21:20:10.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90112025-10-03T21:20:10.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90122025-10-03T21:20:10.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90132025-10-03T21:20:10.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90142025-10-03T21:20:10.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90152025-10-03T21:20:10.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90162025-10-03T21:20:10.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90172025-10-03T21:20:10.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90182025-10-03T21:20:10.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90192025-10-03T21:20:10.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90202025-10-03T21:20:10.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90212025-10-03T21:20:10.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90222025-10-03T21:20:10.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90232025-10-03T21:20:10.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
90242025-10-03T21:20:10.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90252025-10-03T21:20:10.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90262025-10-03T21:20:10.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90272025-10-03T21:20:10.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90282025-10-03T21:20:10.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90292025-10-03T21:20:10.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90302025-10-03T21:20:10.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90312025-10-03T21:20:10.860ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
90322025-10-03T21:20:10.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90332025-10-03T21:20:10.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90342025-10-03T21:20:10.866ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
90352025-10-03T21:20:10.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90362025-10-03T21:20:10.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90372025-10-03T21:20:10.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90382025-10-03T21:20:10.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
90392025-10-03T21:20:10.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90402025-10-03T21:20:10.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90412025-10-03T21:20:10.876ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
90422025-10-03T21:20:10.876ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 739eafe1-9762-49a2-8473-cad21db60ab5, kind: Service { id: 486849a9-1733-4aa9-9b66-43ace2da0eea }, name: Name("external-dns-486849a9-1733-4aa9-9b66-43ace2da0eea"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 10])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f02eae2b-cf44-4f44-9897-f2d5bfac8999, kind: Service { id: 6436aa1b-d53b-419a-817f-0a80ade8ca28 }, name: Name("external-dns-6436aa1b-d53b-419a-817f-0a80ade8ca28"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 189])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 13105992-631f-448a-b9b7-8d11ad5961dd, kind: Service { id: 238731e0-874e-47e4-a037-a86612882fbd }, name: Name("nexus-238731e0-874e-47e4-a037-a86612882fbd"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 84])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d88150f4-7108-4548-9fe5-c31db69daf68, kind: Service { id: a969a278-39b1-4a1a-852e-8c04afad82f6 }, name: Name("nexus-a969a278-39b1-4a1a-852e-8c04afad82f6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 219, 162])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e889ca92-2626-47f4-8ab5-ef4e5457b9a1, kind: Service { id: a5fae325-6f04-4795-bc31-fbf69228d92b }, name: Name("nexus-a5fae325-6f04-4795-bc31-fbf69228d92b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 206, 26])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
90432025-10-03T21:20:10.876ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
90442025-10-03T21:20:10.940ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), time_created: 2025-10-03T21:07:05.994Z, time_modified: 2025-10-03T21:07:05.994Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 3fd3c670-0023-444f-a020-194b49aab89c, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
90452025-10-03T21:20:10.940ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
90462025-10-03T21:20:10.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90472025-10-03T21:20:10.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90482025-10-03T21:20:10.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90492025-10-03T21:20:10.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90502025-10-03T21:20:10.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90512025-10-03T21:20:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 87c00105-6187-43fb-ab98-0bf5dda28bd9 (sled), lookup_type: ById(87c00105-6187-43fb-ab98-0bf5dda28bd9) } result = Ok(())
90522025-10-03T21:20:10.977ZDEBGnexus (BackgroundTasks): client request SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
90532025-10-03T21:20:10.978ZDEBGnexus (BackgroundTasks): client response SledAgent = 87c00105-6187-43fb-ab98-0bf5dda28bd9 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "ff098b9f-5418-452b-9962-8b1afe4bf47b", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
90542025-10-03T21:20:10.980ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
90552025-10-03T21:20:10.980ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = Dns step id = Ensure
90562025-10-03T21:20:10.980ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
90572025-10-03T21:20:10.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90582025-10-03T21:20:10.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90592025-10-03T21:20:10.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90602025-10-03T21:20:10.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90612025-10-03T21:20:10.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90622025-10-03T21:20:10.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
90632025-10-03T21:20:11.000ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6af1969b-3bc3-412a-9f74-798e57542529", "date": "Fri, 03 Oct 2025 21:20:10 GMT"} })
90642025-10-03T21:20:11.000ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 3fd3c670-0023-444f-a020-194b49aab89c switch = Switch0
90652025-10-03T21:20:11.000ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 3fd3c670-0023-444f-a020-194b49aab89c uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
90662025-10-03T21:20:11.007ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 3fd3c670-0023-444f-a020-194b49aab89c result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fd5629a-5bd3-4c3e-b63b-7a29a968ddd4", "content-length": "682", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
90672025-10-03T21:20:11.007ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90682025-10-03T21:20:11.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90692025-10-03T21:20:11.013ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90702025-10-03T21:20:11.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90712025-10-03T21:20:11.017ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90722025-10-03T21:20:11.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90732025-10-03T21:20:11.019ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90742025-10-03T21:20:11.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90752025-10-03T21:20:11.026ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
90762025-10-03T21:20:11.026ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
90772025-10-03T21:20:11.026ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
90782025-10-03T21:20:11.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90792025-10-03T21:20:11.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90802025-10-03T21:20:11.027ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 833.913263ms iteration = 27 status = {}
90812025-10-03T21:20:11.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90822025-10-03T21:20:11.031ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
90832025-10-03T21:20:11.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90842025-10-03T21:20:11.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90852025-10-03T21:20:11.034ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
90862025-10-03T21:20:11.034ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
90872025-10-03T21:20:11.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90882025-10-03T21:20:11.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90892025-10-03T21:20:11.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90902025-10-03T21:20:11.039ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
90912025-10-03T21:20:11.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90922025-10-03T21:20:11.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90932025-10-03T21:20:11.041ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 695d9813-bef7-4502-a7cc-a453b69d8ae0 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
90942025-10-03T21:20:11.041ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
90952025-10-03T21:20:11.041ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
90962025-10-03T21:20:11.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90972025-10-03T21:20:11.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90982025-10-03T21:20:11.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90992025-10-03T21:20:11.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91002025-10-03T21:20:11.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91012025-10-03T21:20:11.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
91022025-10-03T21:20:11.047ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
91032025-10-03T21:20:11.047ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
91042025-10-03T21:20:11.047ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
91052025-10-03T21:20:11.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91062025-10-03T21:20:11.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91072025-10-03T21:20:11.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91082025-10-03T21:20:11.051ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
91092025-10-03T21:20:11.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91102025-10-03T21:20:11.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91112025-10-03T21:20:11.091ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
91122025-10-03T21:20:11.091ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 0 version = 2
91132025-10-03T21:20:11.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91142025-10-03T21:20:11.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91152025-10-03T21:20:11.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91162025-10-03T21:20:11.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
91172025-10-03T21:20:11.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91182025-10-03T21:20:11.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91192025-10-03T21:20:11.098ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 3e237f30-583b-488c-8924-98a2a6d62b7a dns_zone_name = oxide.test found_so_far = 4 version = 2
91202025-10-03T21:20:11.098ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
91212025-10-03T21:20:11.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91222025-10-03T21:20:11.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91232025-10-03T21:20:11.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91242025-10-03T21:20:11.101ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
91252025-10-03T21:20:11.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91262025-10-03T21:20:11.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91272025-10-03T21:20:11.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91282025-10-03T21:20:11.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91292025-10-03T21:20:11.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91302025-10-03T21:20:11.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91312025-10-03T21:20:11.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91322025-10-03T21:20:11.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
91332025-10-03T21:20:11.111ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
91342025-10-03T21:20:11.111ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
91352025-10-03T21:20:11.111ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = OmicronZones step id = Cleanup
91362025-10-03T21:20:11.111ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = OmicronZones step id = Cleanup
91372025-10-03T21:20:11.111ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = PhysicalDisks step id = Cleanup
91382025-10-03T21:20:11.111ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = Clickhouse step id = Ensure
91392025-10-03T21:20:11.111ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = Clickhouse step id = Ensure
91402025-10-03T21:20:11.143ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
91412025-10-03T21:20:11.148ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "64130b48-418a-486d-aceb-985a51f87b5f", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
91422025-10-03T21:20:11.149ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = SupportBundles step id = Cleanup
91432025-10-03T21:20:11.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91442025-10-03T21:20:11.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91452025-10-03T21:20:11.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91462025-10-03T21:20:11.153ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
91472025-10-03T21:20:11.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91482025-10-03T21:20:11.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91492025-10-03T21:20:11.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91502025-10-03T21:20:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91512025-10-03T21:20:11.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91522025-10-03T21:20:11.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
91532025-10-03T21:20:11.163ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = OmicronZones step id = Cleanup
91542025-10-03T21:20:11.163ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
91552025-10-03T21:20:11.163ZDEBGnexus (BackgroundTasks): re-assign sagas: found Nexus instances background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
91562025-10-03T21:20:11.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91572025-10-03T21:20:11.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91582025-10-03T21:20:11.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91592025-10-03T21:20:11.166ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
91602025-10-03T21:20:11.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91612025-10-03T21:20:11.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91622025-10-03T21:20:11.199ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:54 nexus_zone_ids = []
91632025-10-03T21:20:11.199ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(675bea75-8e2d-4986-86f2-b590a30bb084 (blueprint), false)
91642025-10-03T21:20:11.199ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = Cockroach step id = Ensure
91652025-10-03T21:20:11.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91662025-10-03T21:20:11.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91672025-10-03T21:20:11.211ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
91682025-10-03T21:20:11.211ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3dddf372-3461-4ef3-a2ad-590ad19915c4 step component = MgsUpdates step id = Ensure
91692025-10-03T21:20:11.212ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 41 reason = Dependency
91702025-10-03T21:20:11.212ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Dependency
91712025-10-03T21:20:11.212ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91722025-10-03T21:20:11.212ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
91732025-10-03T21:20:11.212ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.018908021s iteration = 15 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3880,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":254535623,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":254535583,"secs":0}},"event_index":1,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":254552023,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":165288175,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":165288146,"secs":0}},"event_index":2,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":419851439,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":54339583,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":54339543,"secs":0}},"event_index":3,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":474203702,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":100904149,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":100904109,"secs":0}},"event_index":4,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":575118941,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":212191025,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":212190975,"secs":0}},"event_index":5,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":787320497,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":130780266,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":130780236,"secs":0}},"event_index":6,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":918109973,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4340,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4300,"secs":0}},"event_index":7,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":918131453,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2300,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2260,"secs":0}},"event_index":8,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":918136804,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2829,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2789,"secs":0}},"event_index":9,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":918141173,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2100,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2071,"secs":0}},"event_index":10,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":918145324,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38418433,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":38418393,"secs":0}},"event_index":11,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":956569607,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":13416891,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":13416831,"secs":0}},"event_index":12,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":969999409,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":35678743,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":35678713,"secs":0}},"event_index":13,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":5694352,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":12889785,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":12889745,"secs":0}},"event_index":14,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":18598427,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4390,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4360,"secs":0}},"event_index":15,"execution_id":"70ddc3fe-85c6-4f89-99ed-d66371fde7ca","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":18609598,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"675bea75-8e2d-4986-86f2-b590a30bb084"}
91742025-10-03T21:20:11.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91752025-10-03T21:20:11.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
91762025-10-03T21:20:11.213ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91772025-10-03T21:20:11.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91782025-10-03T21:20:11.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
91792025-10-03T21:20:11.216ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
91802025-10-03T21:20:11.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
91812025-10-03T21:20:11.216ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
91822025-10-03T21:20:11.216ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91832025-10-03T21:20:11.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91842025-10-03T21:20:11.219ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
91852025-10-03T21:20:11.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
91862025-10-03T21:20:11.223ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91872025-10-03T21:20:11.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
91882025-10-03T21:20:11.225ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
91892025-10-03T21:20:11.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
91902025-10-03T21:20:11.227ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:403
91912025-10-03T21:20:11.227ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 15.641926ms iteration = 41 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
91922025-10-03T21:20:11.228ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 495 local_addr = [fd00:1122:3344:101::b]:48707 method = GET remote_addr = [fd00:1122:3344:101::d]:45317 req_id = 345568b2-02b7-4b8a-a83d-7137ee6b7fce response_code = 200 uri = /a5fae325-6f04-4795-bc31-fbf69228d92b
91932025-10-03T21:20:11.283ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(c5bd80c2-eb4a-4bc4-80c2-5566402782b5, 0), (ce03add5-1108-49b7-ab5b-baeeab316a5f, 0), (69ec4344-f92b-4710-adb9-58137efc8a31, 0), (9fec4929-f241-42c8-a575-34047b4c6206, 0)] collection_id = c5bd80c2-eb4a-4bc4-80c2-5566402782b5
91942025-10-03T21:20:11.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91952025-10-03T21:20:11.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91962025-10-03T21:20:11.402ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = c5bd80c2-eb4a-4bc4-80c2-5566402782b5 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
91972025-10-03T21:20:11.402ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91982025-10-03T21:20:11.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91992025-10-03T21:20:11.406ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(ce03add5-1108-49b7-ab5b-baeeab316a5f, 0), (69ec4344-f92b-4710-adb9-58137efc8a31, 0), (9fec4929-f241-42c8-a575-34047b4c6206, 0), (32405174-c122-4568-b76b-b784f3b35147, 0)] collection_id = ce03add5-1108-49b7-ab5b-baeeab316a5f
92002025-10-03T21:20:11.406ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92012025-10-03T21:20:11.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92022025-10-03T21:20:11.529ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = ce03add5-1108-49b7-ab5b-baeeab316a5f file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
92032025-10-03T21:20:11.529ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92042025-10-03T21:20:11.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92052025-10-03T21:20:11.533ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(69ec4344-f92b-4710-adb9-58137efc8a31, 0), (9fec4929-f241-42c8-a575-34047b4c6206, 0), (32405174-c122-4568-b76b-b784f3b35147, 0)]
92062025-10-03T21:20:11.533ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-87c00105-6187-43fb-ab98-0bf5dda28bd9.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 5257, tv_nsec: 19664508 } })
92072025-10-03T21:20:11.566ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("26cdd4d2-d0ac-43df-bf3e-350fd160efec.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41cf4311-4781-41d3-b8bc-181c344891b0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b55c423c-6cde-4185-879d-3a1c4cf01de4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b807e991-1a36-4877-8937-3efb3994dbc2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f185de2c-df39-4e1b-9686-5fbe577b8506.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::3)) }], valid_until: Instant { tv_sec: 5257, tv_nsec: 52185905 } })
92082025-10-03T21:20:11.566ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
92092025-10-03T21:20:11.566ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
92102025-10-03T21:20:11.567ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
92112025-10-03T21:20:11.567ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a9706ec-aef8-47bf-b3f9-afa893fd5d31", "content-length": "476", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92122025-10-03T21:20:11.567ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
92132025-10-03T21:20:11.567ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "458d0b67-0ff7-48b0-bbd7-cead379e9ebf", "content-length": "734", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92142025-10-03T21:20:11.567ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
92152025-10-03T21:20:11.568ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cabad28-1aee-41dd-b209-264b17135c73", "content-length": "111", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92162025-10-03T21:20:11.568ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
92172025-10-03T21:20:11.568ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2eab06c7-558d-484a-bce0-f8db087ba39c", "content-length": "111", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92182025-10-03T21:20:11.568ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
92192025-10-03T21:20:11.569ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "11b61897-a9c1-4c13-9677-c5a740c468ed", "content-length": "163", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92202025-10-03T21:20:11.569ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
92212025-10-03T21:20:11.569ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "43316a8e-cc76-4f6c-9359-c294a3d48ad8", "content-length": "163", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92222025-10-03T21:20:11.569ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
92232025-10-03T21:20:11.570ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1270fe85-a42e-4bd4-b4af-9199cbdb33a5", "content-length": "166", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92242025-10-03T21:20:11.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
92252025-10-03T21:20:11.570ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "761ca12f-7bd9-4753-a9bb-4678bba66372", "content-length": "166", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92262025-10-03T21:20:11.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
92272025-10-03T21:20:11.570ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d7a7821-2d49-429c-8c1e-508a4b376810", "content-length": "702", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92282025-10-03T21:20:11.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
92292025-10-03T21:20:11.570ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c281663-2681-4c8e-a3d0-e884af036207", "content-length": "727", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92302025-10-03T21:20:11.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
92312025-10-03T21:20:11.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63e80379-a089-46e1-b2a1-0f604dc95c12", "content-length": "729", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92322025-10-03T21:20:11.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
92332025-10-03T21:20:11.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ee74347-77de-4a84-a36f-a122423e8887", "content-length": "728", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92342025-10-03T21:20:11.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
92352025-10-03T21:20:11.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "956fb4e0-8785-431e-9af1-94ab8b540e5c", "content-length": "722", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92362025-10-03T21:20:11.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
92372025-10-03T21:20:11.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "9da9c0b6-c2a5-4ab1-b6f2-371be69a770d", "content-length": "10", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92382025-10-03T21:20:11.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
92392025-10-03T21:20:11.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fadfe64-a479-4bcc-bb8e-6b866177e246", "content-length": "144", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92402025-10-03T21:20:11.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
92412025-10-03T21:20:11.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f68b850-414f-4d68-ba70-105d83b620a4", "content-length": "147", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92422025-10-03T21:20:11.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
92432025-10-03T21:20:11.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "04cec0e0-85e2-4e5f-898a-796136fee88d", "content-length": "109", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92442025-10-03T21:20:11.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
92452025-10-03T21:20:11.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2cc98d6-d9a1-46c4-b4d8-d3ca4955ab00", "content-length": "109", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92462025-10-03T21:20:11.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
92472025-10-03T21:20:11.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4a99803-49c8-4ba0-ad77-3ede56afe845", "content-length": "163", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92482025-10-03T21:20:11.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
92492025-10-03T21:20:11.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7b43de8-e1b0-404f-b62c-a08727263ede", "content-length": "163", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92502025-10-03T21:20:11.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
92512025-10-03T21:20:11.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3af77f6a-95af-4a12-8f58-881ec4f5ce88", "content-length": "166", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92522025-10-03T21:20:11.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
92532025-10-03T21:20:11.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "43f574ed-c7da-4df7-8662-8ba5a86f31b7", "content-length": "166", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92542025-10-03T21:20:11.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
92552025-10-03T21:20:11.575ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3449a9d0-6581-436c-a787-da636d67288c", "content-length": "702", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92562025-10-03T21:20:11.575ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
92572025-10-03T21:20:11.575ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4588b13-83de-4a38-a862-28ca73589f29", "content-length": "727", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92582025-10-03T21:20:11.575ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
92592025-10-03T21:20:11.575ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3be771a4-9931-415a-8c46-77f00d0e0c4d", "content-length": "729", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92602025-10-03T21:20:11.575ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
92612025-10-03T21:20:11.575ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "440bfef4-ae22-494f-b3c4-25ee8b6b5941", "content-length": "728", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92622025-10-03T21:20:11.575ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92632025-10-03T21:20:11.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92642025-10-03T21:20:11.577ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92652025-10-03T21:20:11.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
92662025-10-03T21:20:11.579ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92672025-10-03T21:20:11.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92682025-10-03T21:20:11.581ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92692025-10-03T21:20:11.583ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
92702025-10-03T21:20:11.583ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92712025-10-03T21:20:11.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92722025-10-03T21:20:11.689ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
92732025-10-03T21:20:11.689ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
92742025-10-03T21:20:11.737ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "46693734-5179-4979-8173-73c7a1f01115", "content-length": "88862", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92752025-10-03T21:20:11.739ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
92762025-10-03T21:20:11.739ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
92772025-10-03T21:20:11.739ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
92782025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
92792025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
92802025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
92812025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
92822025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
92832025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b786f91b-2ef0-41fa-8b7a-ec51f6d5f53d", "content-length": "64", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92842025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
92852025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "667f341b-d48a-4855-b2db-2890b30d6f15", "content-length": "64", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92862025-10-03T21:20:11.739ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c9cc66d-2b1f-4162-82ca-19024180a53b", "content-length": "64", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92872025-10-03T21:20:11.740ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "240ed04a-3248-482e-a527-3e313bd6da16", "content-length": "64", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92882025-10-03T21:20:11.740ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
92892025-10-03T21:20:11.740ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
92902025-10-03T21:20:11.740ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
92912025-10-03T21:20:11.740ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "78effe05-d7dd-46ed-882d-e38dc2c34a31", "content-length": "64", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92922025-10-03T21:20:11.740ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
92932025-10-03T21:20:11.777ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "53409a7b-24e3-4cc0-bbb3-d4f04a03171e", "content-length": "401583", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92942025-10-03T21:20:11.778ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb2f2165-2e2f-4a54-a8e9-6751ffc79188", "content-length": "411900", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92952025-10-03T21:20:11.778ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb5946c1-0a3f-49a7-8d16-cd1dd37b345a", "content-length": "409077", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92962025-10-03T21:20:11.779ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d17e3d7-97b3-43c8-8eb6-24e351ce91cf", "content-length": "410962", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92972025-10-03T21:20:11.781ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6816119-e553-41b6-9050-0d89a4a317dd", "content-length": "435292", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
92982025-10-03T21:20:11.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
92992025-10-03T21:20:11.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
93002025-10-03T21:20:11.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
93012025-10-03T21:20:11.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
93022025-10-03T21:20:11.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
93032025-10-03T21:20:11.814ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = f8952d36-0c9f-4327-9fce-f8d697d4e626 (service)
93042025-10-03T21:20:11.814ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
93052025-10-03T21:20:11.817ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "9cd8aae4-a603-4a9d-8b53-941f20f576ca", "content-length": "123", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
93062025-10-03T21:20:11.819ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
93072025-10-03T21:20:11.910ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
93082025-10-03T21:20:11.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
93092025-10-03T21:20:11.910ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e71c02d1-3233-4214-8c28-8efece1d7639", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
93102025-10-03T21:20:11.910ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2efcfb05-0c3d-45aa-b120-c334f3706be4 (service)
93112025-10-03T21:20:11.912ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
93122025-10-03T21:20:11.912ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
93132025-10-03T21:20:11.912ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a25c7765-1f42-4141-b677-5a63139911f9", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
93142025-10-03T21:20:11.912ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9a7c1656-d513-4cc3-b2b0-8e31ad2b5545 (service)
93152025-10-03T21:20:11.913ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
93162025-10-03T21:20:11.913ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
93172025-10-03T21:20:11.914ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e66b5f8e-dc14-42e6-a801-a117cf7efef9", "content-length": "9552", "date": "Fri, 03 Oct 2025 21:20:11 GMT"} })
93182025-10-03T21:20:11.914ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = cac0d292-f20b-4f3a-b918-a96424ce10fa (service)
93192025-10-03T21:20:11.915ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
93202025-10-03T21:20:11.915ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
93212025-10-03T21:20:11.916ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
93222025-10-03T21:20:11.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
93232025-10-03T21:20:11.919ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93242025-10-03T21:20:11.921ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
93252025-10-03T21:20:11.921ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
93262025-10-03T21:20:11.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
93272025-10-03T21:20:12.125ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 8328b657-5c46-473d-bd4e-16a9afa548fc file = nexus/db-queries/src/db/datastore/inventory.rs:1614
93282025-10-03T21:20:12.125ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 8328b657-5c46-473d-bd4e-16a9afa548fc time_started = 2025-10-03 21:20:11.566960 UTC
93292025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 32 reason = Dependency
93302025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 19 reason = Dependency
93312025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 43 reason = Dependency
93322025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93332025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93342025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93352025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 913.990757ms iteration = 16 status = {"collection_id":"8328b657-5c46-473d-bd4e-16a9afa548fc","time_done":"2025-10-03 21:20:11.915174 UTC","time_started":"2025-10-03 21:20:11.566960 UTC"}
93362025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93372025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93382025-10-03T21:20:12.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93392025-10-03T21:20:12.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93402025-10-03T21:20:12.128ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93412025-10-03T21:20:12.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
93422025-10-03T21:20:12.131ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93432025-10-03T21:20:12.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93442025-10-03T21:20:12.132ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
93452025-10-03T21:20:12.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93462025-10-03T21:20:12.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93472025-10-03T21:20:12.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93482025-10-03T21:20:12.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93492025-10-03T21:20:12.136ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93502025-10-03T21:20:12.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93512025-10-03T21:20:12.138ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93522025-10-03T21:20:12.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
93532025-10-03T21:20:12.140ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93542025-10-03T21:20:12.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93552025-10-03T21:20:12.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93562025-10-03T21:20:12.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93572025-10-03T21:20:12.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93582025-10-03T21:20:12.180ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
93592025-10-03T21:20:12.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93602025-10-03T21:20:12.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93612025-10-03T21:20:12.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93622025-10-03T21:20:12.183ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
93632025-10-03T21:20:12.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93642025-10-03T21:20:12.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93652025-10-03T21:20:12.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93662025-10-03T21:20:12.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93672025-10-03T21:20:12.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93682025-10-03T21:20:12.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93692025-10-03T21:20:12.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93702025-10-03T21:20:12.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
93712025-10-03T21:20:12.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93722025-10-03T21:20:12.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93732025-10-03T21:20:12.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93742025-10-03T21:20:12.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93752025-10-03T21:20:12.192ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
93762025-10-03T21:20:12.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93772025-10-03T21:20:12.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93782025-10-03T21:20:12.193ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
93792025-10-03T21:20:12.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93802025-10-03T21:20:12.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93812025-10-03T21:20:12.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93822025-10-03T21:20:12.195ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
93832025-10-03T21:20:12.195ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 69.740447ms iteration = 19 status = {"blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","inventory_collection_id":"39f8892b-4630-448f-9ba3-fb9f3c8b73c2","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
93842025-10-03T21:20:12.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
93852025-10-03T21:20:12.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93862025-10-03T21:20:12.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93872025-10-03T21:20:12.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93882025-10-03T21:20:12.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93892025-10-03T21:20:12.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93902025-10-03T21:20:12.204ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
93912025-10-03T21:20:12.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93922025-10-03T21:20:12.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93932025-10-03T21:20:12.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93942025-10-03T21:20:12.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
93952025-10-03T21:20:12.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93962025-10-03T21:20:12.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93972025-10-03T21:20:12.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93982025-10-03T21:20:12.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93992025-10-03T21:20:12.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94002025-10-03T21:20:12.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94012025-10-03T21:20:12.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94022025-10-03T21:20:12.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
94032025-10-03T21:20:12.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94042025-10-03T21:20:12.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94052025-10-03T21:20:12.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94062025-10-03T21:20:12.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94072025-10-03T21:20:12.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94082025-10-03T21:20:12.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
94092025-10-03T21:20:12.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94102025-10-03T21:20:12.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94112025-10-03T21:20:12.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94122025-10-03T21:20:12.230ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
94132025-10-03T21:20:12.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94142025-10-03T21:20:12.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94152025-10-03T21:20:12.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94162025-10-03T21:20:12.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94172025-10-03T21:20:12.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94182025-10-03T21:20:12.236ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8eb9fc3a-e952-4d43-b095-0b35a6b8ad83, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
94192025-10-03T21:20:12.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94202025-10-03T21:20:12.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94212025-10-03T21:20:12.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94222025-10-03T21:20:12.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94232025-10-03T21:20:12.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94242025-10-03T21:20:12.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94252025-10-03T21:20:12.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94262025-10-03T21:20:12.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
94272025-10-03T21:20:12.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94282025-10-03T21:20:12.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94292025-10-03T21:20:12.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94302025-10-03T21:20:12.247ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
94312025-10-03T21:20:12.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94322025-10-03T21:20:12.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94332025-10-03T21:20:12.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94342025-10-03T21:20:12.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94352025-10-03T21:20:12.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94362025-10-03T21:20:12.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94372025-10-03T21:20:12.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94382025-10-03T21:20:12.255ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 129.37946ms iteration = 43 status = {"physical_disks_added":0}
94392025-10-03T21:20:12.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
94402025-10-03T21:20:12.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94412025-10-03T21:20:12.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94422025-10-03T21:20:12.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94432025-10-03T21:20:12.259ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 6e64322c-d10b-4ebf-9613-4108e59a4368, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
94442025-10-03T21:20:12.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94452025-10-03T21:20:12.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94462025-10-03T21:20:12.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94472025-10-03T21:20:12.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94482025-10-03T21:20:12.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94492025-10-03T21:20:12.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94502025-10-03T21:20:12.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94512025-10-03T21:20:12.267ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
94522025-10-03T21:20:12.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94532025-10-03T21:20:12.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94542025-10-03T21:20:12.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94552025-10-03T21:20:12.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94562025-10-03T21:20:12.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94572025-10-03T21:20:12.272ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
94582025-10-03T21:20:12.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94592025-10-03T21:20:12.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94602025-10-03T21:20:12.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94612025-10-03T21:20:12.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94622025-10-03T21:20:12.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94632025-10-03T21:20:12.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
94642025-10-03T21:20:12.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94652025-10-03T21:20:12.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94662025-10-03T21:20:12.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94672025-10-03T21:20:12.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94682025-10-03T21:20:12.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94692025-10-03T21:20:12.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
94702025-10-03T21:20:12.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94712025-10-03T21:20:12.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94722025-10-03T21:20:12.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94732025-10-03T21:20:12.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94742025-10-03T21:20:12.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94752025-10-03T21:20:12.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
94762025-10-03T21:20:12.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94772025-10-03T21:20:12.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94782025-10-03T21:20:12.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94792025-10-03T21:20:12.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94802025-10-03T21:20:12.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94812025-10-03T21:20:12.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
94822025-10-03T21:20:12.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94832025-10-03T21:20:12.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94842025-10-03T21:20:12.335ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
94852025-10-03T21:20:12.335ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
94862025-10-03T21:20:12.335ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 8384e71c-e872-4be2-9e60-f67eac5e839d parent_id = 675bea75-8e2d-4986-86f2-b590a30bb084 sled_id = 87c00105-6187-43fb-ab98-0bf5dda28bd9
94872025-10-03T21:20:12.335ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:203 parent_blueprint_id = 675bea75-8e2d-4986-86f2-b590a30bb084
94882025-10-03T21:20:12.335ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 209.90395ms iteration = 32 status = {"Unchanged":{"parent_blueprint_id":"675bea75-8e2d-4986-86f2-b590a30bb084","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 87c00105-6187-43fb-ab98-0bf5dda28bd9: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{}},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
94892025-10-03T21:20:15.194ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 158 reason = Timeout
94902025-10-03T21:20:15.194ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94912025-10-03T21:20:15.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94922025-10-03T21:20:15.196ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94932025-10-03T21:20:15.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
94942025-10-03T21:20:15.199ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94952025-10-03T21:20:15.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94962025-10-03T21:20:15.201ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
94972025-10-03T21:20:15.201ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.782627ms iteration = 158 status = {"config_updated":false}